Jul 23 11:26:25.585749 ip-10-0-140-183 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 11:26:26.039021 ip-10-0-140-183 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 11:26:26.039021 ip-10-0-140-183 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 11:26:26.039021 ip-10-0-140-183 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 11:26:26.039021 ip-10-0-140-183 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 11:26:26.039021 ip-10-0-140-183 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 11:26:26.040169 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.039995    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 11:26:26.048618 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048599    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.048618 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048617    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048621    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048624    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048628    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048631    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048634    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048636    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048639    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048641    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048644    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048646    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048649    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048651    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048654    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048661    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048663    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048669    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048672    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048676    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.048691 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048679    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048683    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048685    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048688    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048690    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048693    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048696    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048699    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048701    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048704    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048707    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048711    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048714    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048717    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048720    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048722    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048725    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048728    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048730    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.049185 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048733    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048735    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048739    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048741    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048744    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048747    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048749    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048751    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048754    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048756    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048758    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048761    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048763    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048766    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048768    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048771    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048774    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048776    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048779    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048781    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.049674 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048784    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048786    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048789    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048791    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048793    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048795    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048798    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048800    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048803    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048805    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048808    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048813    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048816    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048820    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048823    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048826    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048828    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048831    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048833    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.050165 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048836    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048838    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048841    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048843    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048845    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048848    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048850    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048853    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048856    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048859    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048861    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048864    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048867    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048869    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048871    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048874    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048876    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048879    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048903    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048908    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.050635 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.048911    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049671    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049679    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049683    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049687    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049690    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049694    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049697    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049699    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049702    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049704    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049707    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049709    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049712    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049714    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049717    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049719    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049722    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049724    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.051108 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049726    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049730    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049733    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049736    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049738    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049741    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049744    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049747    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049751    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049754    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049757    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049760    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049763    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049765    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049769    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049771    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049774    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049776    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049779    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049781    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.051588 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049784    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049786    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049789    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049791    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049794    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049796    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049799    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049801    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049804    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049806    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049809    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049811    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049814    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049817    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049820    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049822    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049824    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049829    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049832    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.052074 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049835    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049837    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049840    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049842    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049845    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049847    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049850    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049852    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049855    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049857    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049860    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049862    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049864    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049867    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049870    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049872    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049875    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049877    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049880    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049882    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.052554 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049885    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049887    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049889    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049892    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049894    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049897    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049900    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049905    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049907    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049911    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049914    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049917    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049919    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049922    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049924    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049927    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049929    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049931    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049934    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049936    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.053057 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049939    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.049941    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050309    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050313    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050317    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050320    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050323    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050326    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050328    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050331    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050333    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050336    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050339    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050341    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050343    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050346    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050349    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050354    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050357    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050359    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.053589 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050362    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050364    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050367    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050369    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050372    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050375    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050378    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050380    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050383    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050385    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050388    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050390    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050393    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050395    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050397    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050400    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050402    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050405    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050408    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050410    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.054130 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050413    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050416    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050418    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050420    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050423    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050426    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050428    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050430    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050433    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050437    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050440    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050443    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050446    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050448    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050451    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050453    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050456    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050473    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050476    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050479    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050481    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.054752 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050484    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050486    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050489    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050491    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050494    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050496    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050499    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050501    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050504    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050507    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050510    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050513    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050516    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050518    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050523    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050527    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050529    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050532    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.055362 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050535    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050543    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050547    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050549    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050552    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050554    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050557    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050559    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050562    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050564    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050567    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050569    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050572    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050574    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050576    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050579    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050583    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050586    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050588    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.055873 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050591    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050594    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.050597    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050675    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050692    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050699    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050704    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050710    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050713    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050717    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050721    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050724    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050727    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050731    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050735    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050739    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050742    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050744    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050747    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050750    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050753    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050757    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050760    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050763    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 11:26:26.056441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050766    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050769    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050773    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050776    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050779    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050782    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050785    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050788    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050791    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050794    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050797    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050801    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050803    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050806    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050809    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050812    2574 flags.go:64] FLAG: --enable-server="true"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050815    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050820    2574 flags.go:64] FLAG: --event-burst="100"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050822    2574 flags.go:64] FLAG: --event-qps="50"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050825    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050828    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050831    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050838    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050841    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050844    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 11:26:26.057119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050848    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050850    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050853    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050856    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050859    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050862    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050865    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050868    2574 flags.go:64] FLAG: --feature-gates=""
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050871    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050874    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050877    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050880    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050883    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050886    2574 flags.go:64] FLAG: --help="false"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050889    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050892    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050894    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050897    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050900    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050904    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050907    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050909    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050912    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 11:26:26.057817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050915    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050918    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050921    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050924    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050927    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050929    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050933    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050936    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050939    2574 flags.go:64] FLAG: --lock-file=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050942    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050945    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050948    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050953    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050956    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050958    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050961    2574 flags.go:64] FLAG: --logging-format="text"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050964    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050967    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050970    2574 flags.go:64] FLAG: --manifest-url=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050972    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050977    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050979    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050983    2574 flags.go:64] FLAG: --max-pods="110"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050986    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050989    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 11:26:26.058453 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050992    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050994    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.050997    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051000    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051003    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051010    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051013    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051016    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051019    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051022    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051027    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051030    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051033    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051037    2574 flags.go:64] FLAG: --port="10250"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051040    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051043    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05209ba7bb32844a9"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051046    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051049    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051052    2574 flags.go:64] FLAG: --register-node="true"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051055    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051058    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051061    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051064    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051067    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051070    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 11:26:26.059087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051073    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051076    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051079    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051082    2574 flags.go:64] FLAG: --runonce="false"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051085    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051088    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051091    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051093    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051096    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051098    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051101    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051104    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051107    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051110    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051112    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051115    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051118    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051126    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051129    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051134    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051138    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051141    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051149    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051152    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051154    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 11:26:26.059757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051157    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051160    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051163    2574 flags.go:64] FLAG: --v="2"
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051171    2574 flags.go:64] FLAG: --version="false"
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051175    2574 flags.go:64] FLAG: --vmodule=""
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051179    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.051182    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051313    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051318    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051322    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051325    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051327    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051330    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051334    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051338    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051340    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051343    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051346    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051348    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051351    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051353    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.060378 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051356    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051358    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051361    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051363    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051366    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051369    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051373    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051375    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051380    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051382    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051384    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051387    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051390    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051392    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051394    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051397    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051401    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051405    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051407    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.060918 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051410    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051412    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051415    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051417    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051420    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051422    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051424    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051427    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051429    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051431    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051434    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051436    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051439    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051441    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051444    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051446    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051448    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051451    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051454    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051471    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.061386 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051475    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051478    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051481    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051483    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051486    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051488    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051491    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051493    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051496    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051498    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051501    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051503    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051506    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051508    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051511    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051513    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051516    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051518    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051521    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051523    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.062058 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051529    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051532    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051534    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051537    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051539    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051542    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051544    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051546    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051549    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051552    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051554    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051558    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051561    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051566    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051570    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051573    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051576    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051578    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051581    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.062861 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051583    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051586    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051589    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051591    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051594    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051597    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.051599    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.052493    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.059809    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.059843    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.059865    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.059871    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059928    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059934    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.063349 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059937    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059940    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059943    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059946    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059949    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059952    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059954    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059957    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059960    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059962    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059965    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059967    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059970    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059972    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059975    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059977    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059980    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059982    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059985    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059987    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.063729 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059989    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059992    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059994    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059997    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.059999    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060001    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060004    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060008    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060010    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060013    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060015    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060018    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060020    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060023    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060026    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060029    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060031    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060034    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060036    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060039    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.064217 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060041    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060043    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060046    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060048    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060051    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060053    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060056    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060058    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060061    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060063    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060066    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060068    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060070    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060073    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060075    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060077    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060080    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060082    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060085    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060088    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.064750 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060091    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060093    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060096    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060098    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060101    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060103    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060106    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060108    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060111    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060113    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060116    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060118    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060121    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060123    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060125    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060128    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060131    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060133    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060135    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.065257 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060138    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060140    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060144    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060147    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060150    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060153    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060155    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060157    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060160    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060162    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060165    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060167    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060171    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060173    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060178    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060181    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060185    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.065952 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060188    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.060193    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060321    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060327    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060330    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060333    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060336    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060338    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060341    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060343    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060346    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060349    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060352    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060355    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060358    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 11:26:26.066444 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060360    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060364    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060367    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060370    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060373    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060376    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060378    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060381    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060383    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060386    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060389    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060392    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060394    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060397    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060400    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060403    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060406    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060410    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 11:26:26.066843 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060414    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060417    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060420    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060423    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060425    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060428    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060430    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060432    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060435    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060437    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060439    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060443    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060446    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060448    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060451    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060453    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060456    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060458    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060479    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 11:26:26.067358 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060481    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060484    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060486    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060489    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060491    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060494    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060496    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060499    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060501    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060504    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060507    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060510    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060512    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060515    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060518    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060520    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060523    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060526    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060528    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060531    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 11:26:26.067841 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060533    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060536    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060538    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060540    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060543    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060546    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060548    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060550    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060553    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060555    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060558    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060560    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060563    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060565    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060567    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060570    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060573    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060575    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060577    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060580    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 11:26:26.068630 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060583    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060585    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060588    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060590    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060593    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060596    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060598    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060601    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:26.060603    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.060608    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.060732    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.064147    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.065873    2574 server.go:1019] "Starting client certificate rotation"
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.065985    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 11:26:26.069289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.066030    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 11:26:26.069668 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.068733    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 11:26:26.070944 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.070925    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 11:26:26.079106 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.079088    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 23 11:26:26.081596 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.081580    2574 log.go:25] "Validated CRI v1 image API"
Jul 23 11:26:26.082511 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.082496    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 11:26:26.089120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.089099    2574 fs.go:135] Filesystem UUIDs: map[07c687e5-3ee4-496e-804b-6d6be08a2921:/dev/nvme0n1p4 6372dca0-7a40-4a35-9fc0-29d3b8262587:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 23 11:26:26.089187 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.089120    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 11:26:26.094767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.094646    2574 manager.go:216] Machine: {Timestamp:2026-07-23 11:26:26.09402811 +0000 UTC m=+0.379493866 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114442 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22fd44b18357ddcebdee66f5367bf9 SystemUUID:ec22fd44-b183-57dd-cebd-ee66f5367bf9 BootID:52e81f75-545d-44f0-baeb-6086167b26cc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d4:bc:a6:b6:45 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d4:bc:a6:b6:45 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:cd:49:0c:6c:cb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 11:26:26.094767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.094764    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 11:26:26.094870 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.094845    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 11:26:26.096935 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.096894    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 11:26:26.096986 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.096962    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 11:26:26.097155 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.096942    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-183.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 11:26:26.097855 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.097843    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 11:26:26.097889 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.097857    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 11:26:26.097889 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.097865    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 11:26:26.098756 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.098746    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 11:26:26.100658 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.100648    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 11:26:26.104286 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104275    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 11:26:26.104396 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104387    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 11:26:26.104424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104418    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 11:26:26.104450 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104427    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 11:26:26.104450 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104440    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 11:26:26.104450 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104449    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 23 11:26:26.104568 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104482    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 11:26:26.104973 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104957    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 11:26:26.105046 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.104975    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 11:26:26.109619 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.109595    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 11:26:26.112117 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.112099    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 11:26:26.112212 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.112138    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 11:26:26.114156 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114141    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114161    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114170    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114178    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114189    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114197    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114210    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114217    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114227    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 11:26:26.114239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114236    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 11:26:26.114513 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114248    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 11:26:26.114513 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.114256    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 11:26:26.115191 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.115179    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 11:26:26.115237 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.115194    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 11:26:26.117344 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.117311    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-183.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 11:26:26.117453 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.117357    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 11:26:26.117453 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.117357    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 11:26:26.118537 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.118522    2574 server.go:1268] "Started kubelet"
Jul 23 11:26:26.118677 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.118620    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 11:26:26.118725 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.118709    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 11:26:26.118777 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.118712    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 11:26:26.119447 ip-10-0-140-183 systemd[1]: Started Kubernetes Kubelet.
Jul 23 11:26:26.121021 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.121003    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 11:26:26.121021 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.121077    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 11:26:26.124615 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.124597    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 11:26:26.130786 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.130768    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 11:26:26.130786 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.130776    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 11:26:26.130942 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.130877    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.131550 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131526    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 11:26:26.131643 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131533    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 11:26:26.131643 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131640    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 11:26:26.131778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131743    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 11:26:26.131778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131755    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 11:26:26.132101 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131976    2574 factory.go:55] Registering systemd factory
Jul 23 11:26:26.132101 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.131992    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 23 11:26:26.132233 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132208    2574 factory.go:153] Registering CRI-O factory
Jul 23 11:26:26.132233 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132230    2574 factory.go:223] Registration of the crio container factory successfully
Jul 23 11:26:26.132333 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132277    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 11:26:26.132333 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132313    2574 factory.go:103] Registering Raw factory
Jul 23 11:26:26.132333 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132329    2574 manager.go:1196] Started watching for new ooms in manager
Jul 23 11:26:26.133002 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.132979    2574 manager.go:349] Starting recovery of all containers
Jul 23 11:26:26.133932 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.133904    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-183.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 11:26:26.134050 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.134033    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 11:26:26.134792 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.133547    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-183.ec2.internal.18c4e7968b956956  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-183.ec2.internal,UID:ip-10-0-140-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-183.ec2.internal,},FirstTimestamp:2026-07-23 11:26:26.11849455 +0000 UTC m=+0.403960309,LastTimestamp:2026-07-23 11:26:26.11849455 +0000 UTC m=+0.403960309,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-183.ec2.internal,}"
Jul 23 11:26:26.144387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.144290    2574 manager.go:354] Recovery completed
Jul 23 11:26:26.148712 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.148699    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.148957 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.148938    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkxkz"
Jul 23 11:26:26.150679 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.150661    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.150746 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.150693    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.150746 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.150708    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.151276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151263    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 11:26:26.151276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151272    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.151276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151276    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 11:26:26.151395 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151296    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 11:26:26.151976 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151961    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.152058 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.151989    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.152058 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.152003    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.152354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.152342    2574 policy_none.go:49] "None policy: Start"
Jul 23 11:26:26.152403 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.152360    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 11:26:26.152403 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.152374    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 11:26:26.152621 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.152554    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-183.ec2.internal.18c4e7968d807af3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-183.ec2.internal,UID:ip-10-0-140-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-183.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-183.ec2.internal,},FirstTimestamp:2026-07-23 11:26:26.150677235 +0000 UTC m=+0.436142993,LastTimestamp:2026-07-23 11:26:26.150677235 +0000 UTC m=+0.436142993,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-183.ec2.internal,}"
Jul 23 11:26:26.153525 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.153515    2574 policy_none.go:47] "Start"
Jul 23 11:26:26.156883 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.156722    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkxkz"
Jul 23 11:26:26.161211 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.161134    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-183.ec2.internal.18c4e7968d80d405  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-183.ec2.internal,UID:ip-10-0-140-183.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-183.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-183.ec2.internal,},FirstTimestamp:2026-07-23 11:26:26.150700037 +0000 UTC m=+0.436165796,LastTimestamp:2026-07-23 11:26:26.150700037 +0000 UTC m=+0.436165796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-183.ec2.internal,}"
Jul 23 11:26:26.190982 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.190967    2574 manager.go:337] "Starting Device Plugin manager"
Jul 23 11:26:26.191083 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.190999    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 11:26:26.191083 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191010    2574 server.go:87] "Starting device plugin registration server"
Jul 23 11:26:26.191276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191264    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 11:26:26.191316 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191280    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 11:26:26.191413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191399    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 11:26:26.192029 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191498    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 11:26:26.192029 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.191509    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 11:26:26.192029 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.192022    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 11:26:26.192173 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.192073    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.290025 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.289928    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.291267    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.291307    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.291332    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.291371    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.291415    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.292296    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.292336    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.292350    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.293144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.292385    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.293943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.293925    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 11:26:26.301129 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.301109    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.301247 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.301136    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-183.ec2.internal\": node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.344195 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.344166    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.391925 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.391880    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"]
Jul 23 11:26:26.392028 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.391983    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.393028 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.393010    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.393109 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.393040    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.393109 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.393050    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.394302 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.394290    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.394421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.394408    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.394455 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.394446    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.395060 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395047    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.395120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395064    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.395120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395073    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.395120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395083    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.395120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395085    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.395120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.395092    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.396204 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.396187    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.396249 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.396221    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:26.396962 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.396939    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:26.396962 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.396962    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:26.397103 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.396976    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:26.418074 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.418051    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-183.ec2.internal\" not found" node="ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.422292 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.422276    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-183.ec2.internal\" not found" node="ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.433910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.433892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.433995 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.433931    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.433995 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.433976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50bdc59f0c097001cebef932b3a8af10-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"50bdc59f0c097001cebef932b3a8af10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.445049 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.445023    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.534533 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.534533 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.534735 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.534735 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50bdc59f0c097001cebef932b3a8af10-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"50bdc59f0c097001cebef932b3a8af10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.534735 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534615    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50bdc59f0c097001cebef932b3a8af10-config\") pod \"kube-apiserver-proxy-ip-10-0-140-183.ec2.internal\" (UID: \"50bdc59f0c097001cebef932b3a8af10\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.534735 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.534644    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/54dc6c6c4449d1b3101c523bac623027-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal\" (UID: \"54dc6c6c4449d1b3101c523bac623027\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.545636 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.545562    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.646409 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.646364    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.720575 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.720547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.724902 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:26.724677    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:26.746745 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.746714    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.847288 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.847198    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:26.947827 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:26.947785    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.048536 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.048503    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.064654 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.064629    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 11:26:27.064778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.064762    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 11:26:27.147353 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.147299    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 11:26:27.148709 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.148685    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.162928 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.162882    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 11:21:26 +0000 UTC" deadline="2028-03-10 17:25:15.118011579 +0000 UTC"
Jul 23 11:26:27.162928 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.162924    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14309h58m47.955091224s"
Jul 23 11:26:27.202235 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.202205    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 11:26:27.203556 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.203538    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 11:26:27.203639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.203578    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 11:26:27.203639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.203588    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeHasSufficientPID"
Jul 23 11:26:27.234916 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:27.234871    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54dc6c6c4449d1b3101c523bac623027.slice/crio-a43b24ccbefb55087bd6f61209d7c7059f044e518c6fdea34f1404b57d2f4d55 WatchSource:0}: Error finding container a43b24ccbefb55087bd6f61209d7c7059f044e518c6fdea34f1404b57d2f4d55: Status 404 returned error can't find the container with id a43b24ccbefb55087bd6f61209d7c7059f044e518c6fdea34f1404b57d2f4d55
Jul 23 11:26:27.240437 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:27.240417    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bdc59f0c097001cebef932b3a8af10.slice/crio-14dd16c8877ce8dc69bcddb0eadf4eed3fe380740ade362878f9d943aa19a899 WatchSource:0}: Error finding container 14dd16c8877ce8dc69bcddb0eadf4eed3fe380740ade362878f9d943aa19a899: Status 404 returned error can't find the container with id 14dd16c8877ce8dc69bcddb0eadf4eed3fe380740ade362878f9d943aa19a899
Jul 23 11:26:27.240437 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.240432    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 11:26:27.249494 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.249454    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.297082 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.297035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" event={"ID":"50bdc59f0c097001cebef932b3a8af10","Type":"ContainerStarted","Data":"14dd16c8877ce8dc69bcddb0eadf4eed3fe380740ade362878f9d943aa19a899"}
Jul 23 11:26:27.298143 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.298117    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"54dc6c6c4449d1b3101c523bac623027","Type":"ContainerStarted","Data":"a43b24ccbefb55087bd6f61209d7c7059f044e518c6fdea34f1404b57d2f4d55"}
Jul 23 11:26:27.349729 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.349696    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.450291 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.450201    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.550852 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:27.550810    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-183.ec2.internal\" not found"
Jul 23 11:26:27.629098 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.629072    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 11:26:27.631276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.631242    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:27.631635 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.631618    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 11:26:27.641608 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.641588    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 11:26:27.642318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.642307    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal"
Jul 23 11:26:27.654805 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:27.654782    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 11:26:28.105547 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.105445    2574 apiserver.go:52] "Watching apiserver"
Jul 23 11:26:28.115535 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.114477    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 11:26:28.115535 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.115073    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz","openshift-cluster-node-tuning-operator/tuned-6jqjd","openshift-image-registry/node-ca-pg9xf","openshift-multus/multus-additional-cni-plugins-bsjcb","openshift-multus/network-metrics-daemon-4nbxv","openshift-network-diagnostics/network-check-target-mgkmf","openshift-network-operator/iptables-alerter-29nbr","openshift-ovn-kubernetes/ovnkube-node-v2v4p","kube-system/global-pull-secret-syncer-wzbq5","kube-system/konnectivity-agent-n9f5t","kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal","openshift-dns/node-resolver-kbssr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal","openshift-multus/multus-rg5f6"]
Jul 23 11:26:28.117878 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.116843    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.119240 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.119216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.120440 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.120417    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 11:26:28.120692 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.120674    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.120784 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.120730    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.120961 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.120945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.121044 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.120966    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7knnw\""
Jul 23 11:26:28.125466 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125446    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125767    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.126807    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125956    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125992    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q7886\""
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.126014    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9g76b\""
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.125819    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.127078    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.126046    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 11:26:28.127206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.126152    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.127799 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.126139    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.127799 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.126611    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.127799 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.127360    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:28.128787 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.128032    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.129395 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.129296    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.130282 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.130262    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 11:26:28.130486 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.130470    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 11:26:28.131206 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.130971    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 11:26:28.131325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131306    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.131397 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131342    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.131518 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131497    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.131608 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131573    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 11:26:28.131667 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131634    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 11:26:28.131776 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.131952 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.131932    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vk9q9\""
Jul 23 11:26:28.132401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.132086    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.132401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.132273    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.132588 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.132458    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tkvkg\""
Jul 23 11:26:28.132710 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.132689    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.132913 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.132701    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-d2gdp\""
Jul 23 11:26:28.134878 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.134206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.135389 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.134992    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.135389 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.135232    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 11:26:28.135760 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.135741    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 11:26:28.136148 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.136044    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 11:26:28.136148 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.136103    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kl9lq\""
Jul 23 11:26:28.137125 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.136672    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r6bn9\""
Jul 23 11:26:28.138515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.137674    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 11:26:28.138515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.137701    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 11:26:28.138515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.138124    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 11:26:28.138515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.138178    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 11:26:28.139233 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.139210    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9pd9p\""
Jul 23 11:26:28.139721 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.139704    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 11:26:28.139890 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.139779    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 11:26:28.142742 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142714    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-var-lib-kubelet\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.142845 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-sys-fs\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.142845 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-konnectivity-ca\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.142845 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-kubelet\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.143013 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142886    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df7nd\" (UniqueName: \"kubernetes.io/projected/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-kube-api-access-df7nd\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.143013 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.142950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-slash\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.143013 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143008    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-host\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.143170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1793efe-de3b-4383-a449-22ac51434713-ovn-node-metrics-cert\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.143170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143077    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.143170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-modprobe-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.143170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-kubernetes\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.143373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-log-socket\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.143373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.143373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143222    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-conf\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.143373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-etc-selinux\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.143373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143349    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-netns\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143385    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-os-release\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-daemon-config\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143528    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-script-lib\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysconfig\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143606    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.143650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143634    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-socket-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cnibin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143717    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-node-log\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f2feb0a-1fb1-4137-abcb-64b3df097c96-tmp-dir\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-lib-modules\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-tmp\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143898    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8pds\" (UniqueName: \"kubernetes.io/projected/25d554ab-8945-41ab-9689-db366cec7a09-kube-api-access-w8pds\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-socket-dir-parent\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-env-overrides\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.144017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.143988    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-cnibin\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144052    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-os-release\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwk48\" (UniqueName: \"kubernetes.io/projected/a1086489-8fd3-4e04-a8ee-290c842353d7-kube-api-access-rwk48\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144181    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144215    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cni-binary-copy\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-multus-certs\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-etc-kubernetes\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.144469 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-systemd\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.144897 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-host\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.144897 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.144521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hppbv\" (UniqueName: \"kubernetes.io/projected/9f2feb0a-1fb1-4137-abcb-64b3df097c96-kube-api-access-hppbv\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.145283 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-dbus\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.145354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-agent-certs\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.145354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145309    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-system-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.145354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145324    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-etc-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.145354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-kubelet-config\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-k8s-cni-cncf-io\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145386    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57b58291-709a-4b50-8f8f-0f362bbe3ac6-iptables-alerter-script\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145419    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f2feb0a-1fb1-4137-abcb-64b3df097c96-hosts-file\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145434    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-system-cni-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145448    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-etc-tuned\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxhh9\" (UniqueName: \"kubernetes.io/projected/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kube-api-access-nxhh9\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-systemd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145492    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-var-lib-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145537    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145554    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sd6w\" (UniqueName: \"kubernetes.io/projected/57b58291-709a-4b50-8f8f-0f362bbe3ac6-kube-api-access-8sd6w\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.145584 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57b58291-709a-4b50-8f8f-0f362bbe3ac6-host-slash\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-systemd-units\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145635    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-bin\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145649    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-netd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-sys\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5snbt\" (UniqueName: \"kubernetes.io/projected/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-kube-api-access-5snbt\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145745    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-kubelet\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145782    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-ovn\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145858    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2zv\" (UniqueName: \"kubernetes.io/projected/d1793efe-de3b-4383-a449-22ac51434713-kube-api-access-dx2zv\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145909    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-conf-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145932    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-serviceca\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.145957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-registration-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146006    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-device-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.146238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-hostroot\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-config\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcz9g\" (UniqueName: \"kubernetes.io/projected/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-kube-api-access-jcz9g\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146156    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-run\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-netns\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-bin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-multus\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.147093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.146313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.148110 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.148084    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 11:26:28.168619 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.168590    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8f4ww"
Jul 23 11:26:28.178376 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.178348    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8f4ww"
Jul 23 11:26:28.194042 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.193909    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 11:26:28.232471 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.232443    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 11:26:28.246586 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-hostroot\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-config\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcz9g\" (UniqueName: \"kubernetes.io/projected/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-kube-api-access-jcz9g\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246679    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-run\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246706    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-netns\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-bin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.246782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-multus\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-var-lib-kubelet\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-sys-fs\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-konnectivity-ca\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-kubelet\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df7nd\" (UniqueName: \"kubernetes.io/projected/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-kube-api-access-df7nd\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-slash\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.246976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-host\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1793efe-de3b-4383-a449-22ac51434713-ovn-node-metrics-cert\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-modprobe-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-kubernetes\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247081    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-hostroot\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-log-socket\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.247152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-conf\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247179    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-etc-selinux\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-netns\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247256    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-os-release\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-daemon-config\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247331    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-script-lib\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247357    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysconfig\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247420    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-socket-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247445    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cnibin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-node-log\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f2feb0a-1fb1-4137-abcb-64b3df097c96-tmp-dir\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-run\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-lib-modules\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-tmp\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247613    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-netns\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.247967 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247659    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-bin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-host\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8pds\" (UniqueName: \"kubernetes.io/projected/25d554ab-8945-41ab-9689-db366cec7a09-kube-api-access-w8pds\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-socket-dir-parent\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-env-overrides\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247773    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-config\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247792    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-cnibin\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-os-release\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247831    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-cnibin\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwk48\" (UniqueName: \"kubernetes.io/projected/a1086489-8fd3-4e04-a8ee-290c842353d7-kube-api-access-rwk48\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247907    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-os-release\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cni-binary-copy\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.247976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-multus-certs\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248005    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-etc-kubernetes\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-systemd\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248074    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-host\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.248874 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248064    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hppbv\" (UniqueName: \"kubernetes.io/projected/9f2feb0a-1fb1-4137-abcb-64b3df097c96-kube-api-access-hppbv\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248136    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-dbus\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-agent-certs\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-system-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248221    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-etc-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-kubelet-config\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-k8s-cni-cncf-io\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248305    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57b58291-709a-4b50-8f8f-0f362bbe3ac6-iptables-alerter-script\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f2feb0a-1fb1-4137-abcb-64b3df097c96-hosts-file\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-system-cni-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248420    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-etc-tuned\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxhh9\" (UniqueName: \"kubernetes.io/projected/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kube-api-access-nxhh9\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-systemd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-daemon-config\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248510    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-var-lib-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.249858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sd6w\" (UniqueName: \"kubernetes.io/projected/57b58291-709a-4b50-8f8f-0f362bbe3ac6-kube-api-access-8sd6w\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248619    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57b58291-709a-4b50-8f8f-0f362bbe3ac6-host-slash\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248648    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248678    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-systemd-units\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-bin\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-netd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-sys\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5snbt\" (UniqueName: \"kubernetes.io/projected/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-kube-api-access-5snbt\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-kubelet\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-ovn\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248938    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2zv\" (UniqueName: \"kubernetes.io/projected/d1793efe-de3b-4383-a449-22ac51434713-kube-api-access-dx2zv\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-conf-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.248993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-serviceca\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249022    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-registration-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.250717 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-device-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249115    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249161    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-device-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-cni-multus\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249331    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-modprobe-d\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249410    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-kubernetes\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-log-socket\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.249593    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.249669    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:28.749641397 +0000 UTC m=+3.035107149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249747    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-var-lib-kubelet\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.249988    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysctl-conf\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250067    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-sys-fs\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250162    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-etc-selinux\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-netns\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-konnectivity-ca\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250359    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.251639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250425    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-os-release\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250475    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-var-lib-kubelet\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.250934    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-socket-dir-parent\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251188    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-systemd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251238    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-var-lib-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251270    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251539    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/57b58291-709a-4b50-8f8f-0f362bbe3ac6-host-slash\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-dbus\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.251948    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-k8s-cni-cncf-io\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252013    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252037    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a1086489-8fd3-4e04-a8ee-290c842353d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252102    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/14e709f1-29c4-4514-ac9f-35b04391b96a-kubelet-config\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252132    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-socket-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252152    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-systemd-units\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-slash\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-kubelet\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252260    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-run-ovn\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252301    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-netd\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.252421 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.252371    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.252427    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:28.75241069 +0000 UTC m=+3.037876452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252472    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-run-ovn-kubernetes\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.252598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/57b58291-709a-4b50-8f8f-0f362bbe3ac6-iptables-alerter-script\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253178    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-system-cni-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253180    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-multus-conf-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9f2feb0a-1fb1-4137-abcb-64b3df097c96-hosts-file\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-node-log\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253306    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-host-cni-bin\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-env-overrides\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253353    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-lib-modules\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253343    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1086489-8fd3-4e04-a8ee-290c842353d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.253394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-registration-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253407    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-system-cni-dir\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-sys\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-host-run-multus-certs\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-sysconfig\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253456    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253657    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1793efe-de3b-4383-a449-22ac51434713-etc-openvswitch\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-serviceca\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253718    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-etc-systemd\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253734    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f2feb0a-1fb1-4137-abcb-64b3df097c96-tmp-dir\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-etc-kubernetes\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25d554ab-8945-41ab-9689-db366cec7a09-host\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cnibin\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.253945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-cni-binary-copy\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.254312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.254110    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1793efe-de3b-4383-a449-22ac51434713-ovnkube-script-lib\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.258965 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.258929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efcb7ca5-ada5-4b47-9a30-1591948a7f0e-agent-certs\") pod \"konnectivity-agent-n9f5t\" (UID: \"efcb7ca5-ada5-4b47-9a30-1591948a7f0e\") " pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.260790 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.260761    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcz9g\" (UniqueName: \"kubernetes.io/projected/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-kube-api-access-jcz9g\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.261125 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.261054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-etc-tuned\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.263613 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.263067    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/25d554ab-8945-41ab-9689-db366cec7a09-tmp\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.263613 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.263126    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:28.263613 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.263146    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:28.263613 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.263167    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:28.263613 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.263241    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:28.763219549 +0000 UTC m=+3.048685315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:28.264596 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.264570    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1793efe-de3b-4383-a449-22ac51434713-ovn-node-metrics-cert\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.264860 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.264833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hppbv\" (UniqueName: \"kubernetes.io/projected/9f2feb0a-1fb1-4137-abcb-64b3df097c96-kube-api-access-hppbv\") pod \"node-resolver-kbssr\" (UID: \"9f2feb0a-1fb1-4137-abcb-64b3df097c96\") " pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.271368 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxhh9\" (UniqueName: \"kubernetes.io/projected/463ca2af-d62b-4fb6-8baf-9055ee8d2b01-kube-api-access-nxhh9\") pod \"aws-ebs-csi-driver-node-d8zbz\" (UID: \"463ca2af-d62b-4fb6-8baf-9055ee8d2b01\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.271368 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271167    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwk48\" (UniqueName: \"kubernetes.io/projected/a1086489-8fd3-4e04-a8ee-290c842353d7-kube-api-access-rwk48\") pod \"multus-additional-cni-plugins-bsjcb\" (UID: \"a1086489-8fd3-4e04-a8ee-290c842353d7\") " pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.271368 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2zv\" (UniqueName: \"kubernetes.io/projected/d1793efe-de3b-4383-a449-22ac51434713-kube-api-access-dx2zv\") pod \"ovnkube-node-v2v4p\" (UID: \"d1793efe-de3b-4383-a449-22ac51434713\") " pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.271368 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271318    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df7nd\" (UniqueName: \"kubernetes.io/projected/f91f7b99-7303-41ef-a7d7-f04d5db6fea5-kube-api-access-df7nd\") pod \"multus-rg5f6\" (UID: \"f91f7b99-7303-41ef-a7d7-f04d5db6fea5\") " pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.271784 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271685    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8pds\" (UniqueName: \"kubernetes.io/projected/25d554ab-8945-41ab-9689-db366cec7a09-kube-api-access-w8pds\") pod \"tuned-6jqjd\" (UID: \"25d554ab-8945-41ab-9689-db366cec7a09\") " pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.271784 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.271691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sd6w\" (UniqueName: \"kubernetes.io/projected/57b58291-709a-4b50-8f8f-0f362bbe3ac6-kube-api-access-8sd6w\") pod \"iptables-alerter-29nbr\" (UID: \"57b58291-709a-4b50-8f8f-0f362bbe3ac6\") " pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.274084 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.272305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5snbt\" (UniqueName: \"kubernetes.io/projected/64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25-kube-api-access-5snbt\") pod \"node-ca-pg9xf\" (UID: \"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25\") " pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.441860 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.441760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-29nbr"
Jul 23 11:26:28.453370 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.453343    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kbssr"
Jul 23 11:26:28.462284 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.462249    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pg9xf"
Jul 23 11:26:28.470826 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.470792    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bsjcb"
Jul 23 11:26:28.481806 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.481777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz"
Jul 23 11:26:28.488794 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.488765    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6jqjd"
Jul 23 11:26:28.498671 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.498646    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:28.504728 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.504705    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rg5f6"
Jul 23 11:26:28.513004 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.512980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:28.753232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.753145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:28.753232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.753220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:28.753443 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.753340    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:28.753443 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.753350    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:28.753443 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.753425    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:29.753403096 +0000 UTC m=+4.038868858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:28.753443 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.753444    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:29.753433814 +0000 UTC m=+4.038899566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:28.853928 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:28.853884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:28.854128 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.854093    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:28.854128 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.854118    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:28.854128 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.854129    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:28.854237 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:28.854189    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:29.854171854 +0000 UTC m=+4.139637611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:29.181147 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.181049    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 11:21:28 +0000 UTC" deadline="2028-02-16 12:18:57.865090588 +0000 UTC"
Jul 23 11:26:29.181147 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.181080    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13752h52m28.684014651s"
Jul 23 11:26:29.292221 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.292182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:29.292400 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.292327    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:29.422812 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.422550    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1086489_8fd3_4e04_a8ee_290c842353d7.slice/crio-a6cecaae2a73dd35934e6082cc12904502087dc8d096844fb53f63db60ceff7e WatchSource:0}: Error finding container a6cecaae2a73dd35934e6082cc12904502087dc8d096844fb53f63db60ceff7e: Status 404 returned error can't find the container with id a6cecaae2a73dd35934e6082cc12904502087dc8d096844fb53f63db60ceff7e
Jul 23 11:26:29.427799 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.427767    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1793efe_de3b_4383_a449_22ac51434713.slice/crio-aa27070eefac99d78e0c9162a6c59ceae7e9e7e092c0ca9a239b19bcd889310e WatchSource:0}: Error finding container aa27070eefac99d78e0c9162a6c59ceae7e9e7e092c0ca9a239b19bcd889310e: Status 404 returned error can't find the container with id aa27070eefac99d78e0c9162a6c59ceae7e9e7e092c0ca9a239b19bcd889310e
Jul 23 11:26:29.428896 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.428870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25d554ab_8945_41ab_9689_db366cec7a09.slice/crio-8e94d2f6d52e94fbc9b5899a9854efc4a1b5f2202931a0f1643731de14cecb6e WatchSource:0}: Error finding container 8e94d2f6d52e94fbc9b5899a9854efc4a1b5f2202931a0f1643731de14cecb6e: Status 404 returned error can't find the container with id 8e94d2f6d52e94fbc9b5899a9854efc4a1b5f2202931a0f1643731de14cecb6e
Jul 23 11:26:29.429804 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.429776    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463ca2af_d62b_4fb6_8baf_9055ee8d2b01.slice/crio-6ebd372154cacaf62e64f72c5b5b0b02ae960374a9cd3d1b9754f62ad28f3518 WatchSource:0}: Error finding container 6ebd372154cacaf62e64f72c5b5b0b02ae960374a9cd3d1b9754f62ad28f3518: Status 404 returned error can't find the container with id 6ebd372154cacaf62e64f72c5b5b0b02ae960374a9cd3d1b9754f62ad28f3518
Jul 23 11:26:29.430527 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.430500    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91f7b99_7303_41ef_a7d7_f04d5db6fea5.slice/crio-e7a831df504e02b0b9578fb6987aa21a89f18c07c3857006c01239bea3dd06ef WatchSource:0}: Error finding container e7a831df504e02b0b9578fb6987aa21a89f18c07c3857006c01239bea3dd06ef: Status 404 returned error can't find the container with id e7a831df504e02b0b9578fb6987aa21a89f18c07c3857006c01239bea3dd06ef
Jul 23 11:26:29.432789 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.432715    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f2feb0a_1fb1_4137_abcb_64b3df097c96.slice/crio-91eb8d0362aea9e65ca4a652fe108af7120425352ac0c251f309f83fa67e7adb WatchSource:0}: Error finding container 91eb8d0362aea9e65ca4a652fe108af7120425352ac0c251f309f83fa67e7adb: Status 404 returned error can't find the container with id 91eb8d0362aea9e65ca4a652fe108af7120425352ac0c251f309f83fa67e7adb
Jul 23 11:26:29.433522 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.433486    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcb7ca5_ada5_4b47_9a30_1591948a7f0e.slice/crio-dbe919d1442681e199a4156db76fdf3e723cb432488b7ecc659d8f8a8d417b3f WatchSource:0}: Error finding container dbe919d1442681e199a4156db76fdf3e723cb432488b7ecc659d8f8a8d417b3f: Status 404 returned error can't find the container with id dbe919d1442681e199a4156db76fdf3e723cb432488b7ecc659d8f8a8d417b3f
Jul 23 11:26:29.436840 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:26:29.436740    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64cf79fc_0c55_4ab7_8d1f_b41c47e5cc25.slice/crio-b8dec4f4313f3e91671b7e3e70843ad874576f9dad27f134877eb716e61bd01a WatchSource:0}: Error finding container b8dec4f4313f3e91671b7e3e70843ad874576f9dad27f134877eb716e61bd01a: Status 404 returned error can't find the container with id b8dec4f4313f3e91671b7e3e70843ad874576f9dad27f134877eb716e61bd01a
Jul 23 11:26:29.762066 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.761865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:29.762252 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.762090    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:29.762252 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.762030    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:29.762252 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.762198    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:29.762252 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.762200    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:31.76218372 +0000 UTC m=+6.047649467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:29.762252 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.762250    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:31.762236112 +0000 UTC m=+6.047701861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:29.863031 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:29.862995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:29.863181 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.863159    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:29.863239 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.863181    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:29.863239 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.863192    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:29.863312 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:29.863246    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:31.863229573 +0000 UTC m=+6.148695320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:30.181882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.181747    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 11:21:28 +0000 UTC" deadline="2028-02-19 12:41:31.901218274 +0000 UTC"
Jul 23 11:26:30.181882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.181815    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13825h15m1.719407146s"
Jul 23 11:26:30.294073 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.294034    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:30.294248 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:30.294203    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:30.294313 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.294291    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:30.294379 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:30.294367    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:30.311626 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.311585    2574 generic.go:356] "Generic (PLEG): container finished" podID="54dc6c6c4449d1b3101c523bac623027" containerID="481d54f5f083c23a78121e901fe6820cb7d0fd1f0ad78beb0e0c93bed03df65d" exitCode=0
Jul 23 11:26:30.312262 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.312235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"54dc6c6c4449d1b3101c523bac623027","Type":"ContainerDied","Data":"481d54f5f083c23a78121e901fe6820cb7d0fd1f0ad78beb0e0c93bed03df65d"}
Jul 23 11:26:30.318719 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.318684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9f5t" event={"ID":"efcb7ca5-ada5-4b47-9a30-1591948a7f0e","Type":"ContainerStarted","Data":"dbe919d1442681e199a4156db76fdf3e723cb432488b7ecc659d8f8a8d417b3f"}
Jul 23 11:26:30.329003 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.328960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kbssr" event={"ID":"9f2feb0a-1fb1-4137-abcb-64b3df097c96","Type":"ContainerStarted","Data":"91eb8d0362aea9e65ca4a652fe108af7120425352ac0c251f309f83fa67e7adb"}
Jul 23 11:26:30.332125 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.332093    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz" event={"ID":"463ca2af-d62b-4fb6-8baf-9055ee8d2b01","Type":"ContainerStarted","Data":"6ebd372154cacaf62e64f72c5b5b0b02ae960374a9cd3d1b9754f62ad28f3518"}
Jul 23 11:26:30.339353 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.339315    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-29nbr" event={"ID":"57b58291-709a-4b50-8f8f-0f362bbe3ac6","Type":"ContainerStarted","Data":"5000392430c369f0dc94586a8087f8a02c8e83dedfb07b956c0d2c28a7ed2b1e"}
Jul 23 11:26:30.350333 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.349620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" event={"ID":"50bdc59f0c097001cebef932b3a8af10","Type":"ContainerStarted","Data":"0de9d395aaab5fe402d93f0d423126a0d19f5a07fac7d50077ed45b782ee4dae"}
Jul 23 11:26:30.358570 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.358461    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pg9xf" event={"ID":"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25","Type":"ContainerStarted","Data":"b8dec4f4313f3e91671b7e3e70843ad874576f9dad27f134877eb716e61bd01a"}
Jul 23 11:26:30.375597 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.375561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rg5f6" event={"ID":"f91f7b99-7303-41ef-a7d7-f04d5db6fea5","Type":"ContainerStarted","Data":"e7a831df504e02b0b9578fb6987aa21a89f18c07c3857006c01239bea3dd06ef"}
Jul 23 11:26:30.386413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.386374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6jqjd" event={"ID":"25d554ab-8945-41ab-9689-db366cec7a09","Type":"ContainerStarted","Data":"8e94d2f6d52e94fbc9b5899a9854efc4a1b5f2202931a0f1643731de14cecb6e"}
Jul 23 11:26:30.389007 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.388905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"aa27070eefac99d78e0c9162a6c59ceae7e9e7e092c0ca9a239b19bcd889310e"}
Jul 23 11:26:30.391680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:30.391649    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerStarted","Data":"a6cecaae2a73dd35934e6082cc12904502087dc8d096844fb53f63db60ceff7e"}
Jul 23 11:26:31.292809 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.292775    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:31.293335 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.292956    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:31.423436 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.422854    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" event={"ID":"54dc6c6c4449d1b3101c523bac623027","Type":"ContainerStarted","Data":"c6a0a247c9047429b22e1a239ec43deca6fb54a2679e5863247220ce8f3997de"}
Jul 23 11:26:31.444439 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.444373    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-183.ec2.internal" podStartSLOduration=4.444346383 podStartE2EDuration="4.444346383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:30.372280472 +0000 UTC m=+4.657746246" watchObservedRunningTime="2026-07-23 11:26:31.444346383 +0000 UTC m=+5.729812151"
Jul 23 11:26:31.783004 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.782910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:31.783175 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.783023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:31.783245 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.783170    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:31.783245 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.783240    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:35.783220165 +0000 UTC m=+10.068685913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:31.783775 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.783682    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:31.783775 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.783746    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:35.783728254 +0000 UTC m=+10.069194004 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:31.884048 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:31.884004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:31.884250 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.884212    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:31.884250 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.884233    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:31.884250 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.884247    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:31.884447 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:31.884309    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:35.884289399 +0000 UTC m=+10.169755169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:32.304387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:32.302625    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:32.304387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:32.302625    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:32.304387 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:32.302827    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:32.304387 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:32.302990    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:33.292400 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:33.292000    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:33.292400 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:33.292114    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:34.293488 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:34.292243    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:34.293488 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:34.292371    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:34.293488 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:34.292734    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:34.293488 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:34.292810    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:35.292247 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:35.292214    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:35.292388 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.292334    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:35.822819 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:35.822781    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:35.823244 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:35.822846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:35.823244 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.822970    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:35.823244 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.823033    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:43.8230134 +0000 UTC m=+18.108479152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:35.823244 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.823101    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:35.823244 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.823136    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:43.823124948 +0000 UTC m=+18.108590708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:35.924778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:35.924067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:35.924778 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.924327    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:35.924778 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.924370    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:35.924778 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.924397    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:35.924778 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:35.924489    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:43.924451838 +0000 UTC m=+18.209917585 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:36.295919 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:36.295838    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:36.296070 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:36.295962    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:36.296336 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:36.296311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:36.296420 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:36.296406    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:37.292289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:37.292229    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:37.292857 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:37.292381    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:38.292381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:38.292294    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:38.292825 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:38.292405    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:38.292825 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:38.292759    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:38.292951 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:38.292845    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:39.291797 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:39.291761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:39.291955 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:39.291898    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:40.291872 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:40.291830    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:40.292325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:40.291830    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:40.292325 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:40.291960    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:40.292325 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:40.292026    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:41.291843 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:41.291807    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:41.292027 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:41.291938    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:42.291750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:42.291720    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:42.291923 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:42.291718    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:42.291923 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:42.291857    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:42.291923 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:42.291919    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:43.292104 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:43.292063    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:43.292602 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.292197    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:43.887608 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:43.887572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:43.887781 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:43.887663    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:43.887781 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.887752    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:43.887900 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.887785    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:43.887900 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.887829    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:26:59.887806346 +0000 UTC m=+34.173272108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:43.887900 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.887851    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:26:59.88783939 +0000 UTC m=+34.173305136 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:43.988220 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:43.988189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:43.988350 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.988305    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:43.988350 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.988317    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:43.988350 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.988326    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:43.988480 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:43.988369    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:26:59.988355914 +0000 UTC m=+34.273821658 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:44.291826 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:44.291554    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:44.291826 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:44.291735    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:44.291826 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:44.291752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:44.292082 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:44.291843    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:45.293290 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:45.292084    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:45.293290 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:45.292241    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:46.292790 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:46.292754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:46.292975 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:46.292878    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:46.292975 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:46.292933    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:46.293092 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:46.293041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:47.292324 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:47.292288    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:47.292857 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:47.292420    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:48.291906 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:48.291870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:48.292201 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:48.291881    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:48.292201 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:48.292018    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:48.292201 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:48.292053    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:49.292620 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.292295    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:49.293350 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:49.292712    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:49.472325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.472291    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pg9xf" event={"ID":"64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25","Type":"ContainerStarted","Data":"65687812546ab4d931362f8bdea8bb614e94b7f78de783774f93ede6ca2d70f2"}
Jul 23 11:26:49.473998 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.473955    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rg5f6" event={"ID":"f91f7b99-7303-41ef-a7d7-f04d5db6fea5","Type":"ContainerStarted","Data":"0bf81c3c6890f47cae12e2f18ea49b1b6e90905735597f7034cfad4e47cfd913"}
Jul 23 11:26:49.475577 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.475549    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6jqjd" event={"ID":"25d554ab-8945-41ab-9689-db366cec7a09","Type":"ContainerStarted","Data":"d094481bd62051ca219a7d14291346a70054cdd4fbe4199b92e328c9050368a4"}
Jul 23 11:26:49.478702 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.478683    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:26:49.479123 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479095    2574 generic.go:356] "Generic (PLEG): container finished" podID="d1793efe-de3b-4383-a449-22ac51434713" containerID="543ffeafc72b3933e9531b8d0d250aadc14281f41b2048c4bfe99e0a5e3e3c39" exitCode=1
Jul 23 11:26:49.479218 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"b0332f1cb10c7f42a600e6dff3ce0b8d6781955eda58dd63956a62893f84696b"}
Jul 23 11:26:49.479285 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479241    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"a4231ad014f8ff44ac1254a11bfa7195eb14a999cafc05c97b98d64bb175c84b"}
Jul 23 11:26:49.479337 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479295    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"f3c186eb5ec991830c36aeba0eb084fbfef7a58b5ccb5e548b384a62bf7261c2"}
Jul 23 11:26:49.479394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"664ac37d0de1c324d6c956f247c4366b9c6ea26c3361d27c07fc3889651d61e7"}
Jul 23 11:26:49.479447 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479395    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerDied","Data":"543ffeafc72b3933e9531b8d0d250aadc14281f41b2048c4bfe99e0a5e3e3c39"}
Jul 23 11:26:49.479527 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.479447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"165683a3e280a69811d40b95d85574fcf98ff87794c8da93228acfde2de58fe2"}
Jul 23 11:26:49.480973 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.480938    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="3ddd4ef42a89a182b35415d3c466f24db6e6e2644d769122314ee97432dfa3b4" exitCode=0
Jul 23 11:26:49.481071 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.480998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"3ddd4ef42a89a182b35415d3c466f24db6e6e2644d769122314ee97432dfa3b4"}
Jul 23 11:26:49.482806 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.482783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9f5t" event={"ID":"efcb7ca5-ada5-4b47-9a30-1591948a7f0e","Type":"ContainerStarted","Data":"00042fb795a6bb56665b6b86ff3c343b48c3495ba47579d6322298d85be4e996"}
Jul 23 11:26:49.484718 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.484699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kbssr" event={"ID":"9f2feb0a-1fb1-4137-abcb-64b3df097c96","Type":"ContainerStarted","Data":"c8462c383a211db796e467d3c29f420ab28d002c67e7661de377f7daed066279"}
Jul 23 11:26:49.486357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.486334    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz" event={"ID":"463ca2af-d62b-4fb6-8baf-9055ee8d2b01","Type":"ContainerStarted","Data":"489093b87e4c0d8a49e3c827e4e324e57ea03377cc751a8e0293ecd873057837"}
Jul 23 11:26:49.494327 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.494288    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pg9xf" podStartSLOduration=8.955282349 podStartE2EDuration="23.494274849s" totalImagesPullingTime="14.5389925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:49.49394524 +0000 UTC m=+23.779411010" watchObservedRunningTime="2026-07-23 11:26:49.494274849 +0000 UTC m=+23.779740615"
Jul 23 11:26:49.494831 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.494805    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-183.ec2.internal" podStartSLOduration=22.494797143 podStartE2EDuration="22.494797143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:31.445015728 +0000 UTC m=+5.730481499" watchObservedRunningTime="2026-07-23 11:26:49.494797143 +0000 UTC m=+23.780262907"
Jul 23 11:26:49.540000 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.539897    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6jqjd" podStartSLOduration=4.454524658 podStartE2EDuration="23.539874294s" totalImagesPullingTime="19.085349636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:49.539857784 +0000 UTC m=+23.825323576" watchObservedRunningTime="2026-07-23 11:26:49.539874294 +0000 UTC m=+23.825340061"
Jul 23 11:26:49.554149 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.554096    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n9f5t" podStartSLOduration=4.500325376 podStartE2EDuration="23.554082794s" totalImagesPullingTime="19.053757418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:49.553631804 +0000 UTC m=+23.839097569" watchObservedRunningTime="2026-07-23 11:26:49.554082794 +0000 UTC m=+23.839548559"
Jul 23 11:26:49.585282 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.583729    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kbssr" podStartSLOduration=4.529332391 podStartE2EDuration="23.583708212s" totalImagesPullingTime="19.054375821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:49.58294509 +0000 UTC m=+23.868410877" watchObservedRunningTime="2026-07-23 11:26:49.583708212 +0000 UTC m=+23.869173978"
Jul 23 11:26:49.609423 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.609367    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rg5f6" podStartSLOduration=4.33675049 podStartE2EDuration="23.609350855s" totalImagesPullingTime="19.272600365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:49.60923411 +0000 UTC m=+23.894699876" watchObservedRunningTime="2026-07-23 11:26:49.609350855 +0000 UTC m=+23.894816621"
Jul 23 11:26:49.869402 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:49.869371    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 11:26:50.203651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.203479    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T11:26:49.869396433Z","UUID":"3e65a02c-b403-4128-b765-550da05c707f","Handler":null,"Name":"","Endpoint":""}
Jul 23 11:26:50.205176 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.205154    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 11:26:50.205305 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.205201    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 11:26:50.291877 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.291848    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:50.291877 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.291863    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:50.292119 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:50.291980    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:50.292119 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:50.292095    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:50.491095 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.491009    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz" event={"ID":"463ca2af-d62b-4fb6-8baf-9055ee8d2b01","Type":"ContainerStarted","Data":"e83a45fe461cbaae97b0b2b6ba822bd3c61c5c84e873ffc7f29724f043415351"}
Jul 23 11:26:50.493334 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.493302    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-29nbr" event={"ID":"57b58291-709a-4b50-8f8f-0f362bbe3ac6","Type":"ContainerStarted","Data":"b27fd2130d8b3280d9ba6b78655a0b39472d285a383c49757dc5f340b9868f3e"}
Jul 23 11:26:50.512249 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.512186    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-29nbr" podStartSLOduration=5.399391635 podStartE2EDuration="24.512164247s" totalImagesPullingTime="19.112772612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:50.511997542 +0000 UTC m=+24.797463309" watchObservedRunningTime="2026-07-23 11:26:50.512164247 +0000 UTC m=+24.797630014"
Jul 23 11:26:50.658541 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.658502    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:50.659266 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:50.659240    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:51.292019 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.291976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:51.292214 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:51.292101    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:51.498764 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.498544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz" event={"ID":"463ca2af-d62b-4fb6-8baf-9055ee8d2b01","Type":"ContainerStarted","Data":"87fb5276ec39b8f4e845421befb2ebe553f3abed8f0930dc0b21f62ae1143154"}
Jul 23 11:26:51.502254 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.502226    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:26:51.505684 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.503152    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"2b6d9a5b3a24d2e54034955f419cd54cf9ed1391ec09271a2875de9443a39dee"}
Jul 23 11:26:51.505684 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.503948    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:51.505684 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.504498    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n9f5t"
Jul 23 11:26:51.519631 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:51.519581    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d8zbz" podStartSLOduration=4.793615971 podStartE2EDuration="25.519565173s" totalImagesPullingTime="20.725949202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:51.51953148 +0000 UTC m=+25.804997255" watchObservedRunningTime="2026-07-23 11:26:51.519565173 +0000 UTC m=+25.805030933"
Jul 23 11:26:52.291841 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:52.291783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:52.292034 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:52.291784    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:52.292034 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:52.291916    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:52.292117 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:52.292031    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:53.292277 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:53.292240    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:53.292981 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:53.292395    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:54.292495 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.292445    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:54.293197 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:54.292619    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:54.293197 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.292658    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:54.293197 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:54.292753    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:54.514124 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.513892    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:26:54.514634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.514608    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"ba0e6151f440977478b8fbe741f9ec6aa9e6f686c6af31716e2e3f322c1503bb"}
Jul 23 11:26:54.515056 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.515041    2574 scope.go:117] "RemoveContainer" containerID="543ffeafc72b3933e9531b8d0d250aadc14281f41b2048c4bfe99e0a5e3e3c39"
Jul 23 11:26:54.515351 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.515337    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:54.515506 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.515495    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:54.515540 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.515536    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:54.532182 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.532158    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:54.533635 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:54.533614    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:26:55.292393 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.292360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:55.292590 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:55.292508    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:55.519855 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.519781    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:26:55.520146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.520123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" event={"ID":"d1793efe-de3b-4383-a449-22ac51434713","Type":"ContainerStarted","Data":"f755d380b83dcea3ac85b2af1cabafc761151ff1f247715f8304ece475468ab4"}
Jul 23 11:26:55.521848 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.521823    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="e48d19423a1ac524569ad23777f528eb70e9ca3320e6c3b679d15d5496bcd4ea" exitCode=0
Jul 23 11:26:55.521938 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.521872    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"e48d19423a1ac524569ad23777f528eb70e9ca3320e6c3b679d15d5496bcd4ea"}
Jul 23 11:26:55.547952 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:55.547904    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p" podStartSLOduration=9.378716092 podStartE2EDuration="28.547888761s" totalImagesPullingTime="19.169172669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:26:55.547677735 +0000 UTC m=+29.833143501" watchObservedRunningTime="2026-07-23 11:26:55.547888761 +0000 UTC m=+29.833354526"
Jul 23 11:26:56.293239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.293208    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:56.293771 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:56.293309    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:56.293771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.293359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:56.293771 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:56.293452    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:56.404047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.404018    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wzbq5"]
Jul 23 11:26:56.404204 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.404133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:56.404250 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:56.404217    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:56.408502 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.408446    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4nbxv"]
Jul 23 11:26:56.409097 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.409076    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mgkmf"]
Jul 23 11:26:56.524409 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.524379    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:56.524579 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:56.524389    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:56.524579 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:56.524523    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:56.524579 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:56.524568    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:57.529009 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:57.528790    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="3634506f066836139d4a61521cfa0f70203183cdd6735bd9620f84b8dfaa8f24" exitCode=0
Jul 23 11:26:57.529406 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:57.528874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"3634506f066836139d4a61521cfa0f70203183cdd6735bd9620f84b8dfaa8f24"}
Jul 23 11:26:58.291917 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:58.291875    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:58.292087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:58.291876    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:58.292087 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:58.292025    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:26:58.292087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:58.292060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:58.292206 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:58.292141    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:26:58.292242 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:58.292216    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:26:58.533980 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:58.533878    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="a98911d4a9b19fafc298f928d9ef786dbad94a3ff90ad182685fb3b739497040" exitCode=0
Jul 23 11:26:58.533980 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:58.533935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"a98911d4a9b19fafc298f928d9ef786dbad94a3ff90ad182685fb3b739497040"}
Jul 23 11:26:59.896354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:59.896319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:26:59.896905 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:59.896406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:26:59.896905 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.896503    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:59.896905 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.896569    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs podName:ebe0b363-e249-4eb8-a373-8ab4ad5a5f11 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:31.896550567 +0000 UTC m=+66.182016310 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs") pod "network-metrics-daemon-4nbxv" (UID: "ebe0b363-e249-4eb8-a373-8ab4ad5a5f11") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 11:26:59.896905 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.896506    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:59.896905 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.896669    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret podName:14e709f1-29c4-4514-ac9f-35b04391b96a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:31.896647857 +0000 UTC m=+66.182113615 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret") pod "global-pull-secret-syncer-wzbq5" (UID: "14e709f1-29c4-4514-ac9f-35b04391b96a") : object "kube-system"/"original-pull-secret" not registered
Jul 23 11:26:59.997129 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:26:59.997096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:26:59.997329 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.997242    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 11:26:59.997329 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.997264    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 11:26:59.997329 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.997276    2574 projected.go:196] Error preparing data for projected volume kube-api-access-f9jln for pod openshift-network-diagnostics/network-check-target-mgkmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:26:59.997498 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:26:59.997341    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln podName:fe8651cb-6ed7-433e-b2be-d7ae522dd03e nodeName:}" failed. No retries permitted until 2026-07-23 11:27:31.997323414 +0000 UTC m=+66.282789171 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f9jln" (UniqueName: "kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln") pod "network-check-target-mgkmf" (UID: "fe8651cb-6ed7-433e-b2be-d7ae522dd03e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 11:27:00.292191 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:00.292107    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:00.292191 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:00.292157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:27:00.292414 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:00.292222    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:27:00.292414 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:00.292241    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mgkmf" podUID="fe8651cb-6ed7-433e-b2be-d7ae522dd03e"
Jul 23 11:27:00.292414 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:00.292319    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4nbxv" podUID="ebe0b363-e249-4eb8-a373-8ab4ad5a5f11"
Jul 23 11:27:00.292414 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:00.292402    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wzbq5" podUID="14e709f1-29c4-4514-ac9f-35b04391b96a"
Jul 23 11:27:01.558993 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.558929    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-183.ec2.internal" event="NodeReady"
Jul 23 11:27:01.559405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.559067    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 11:27:01.600634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.600593    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:27:01.614225 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.614197    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.618083 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.618051    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vjptn"]
Jul 23 11:27:01.619231 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.619197    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 11:27:01.619426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.619373    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 11:27:01.619581 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.619452    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 11:27:01.619581 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.619532    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t2g4d\""
Jul 23 11:27:01.631760 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.631732    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 11:27:01.636314 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.636291    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:27:01.636439 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.636325    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5c96j"]
Jul 23 11:27:01.636439 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.636425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.638963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.638937    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 11:27:01.639261 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.639243    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 11:27:01.639515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.639448    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d6qnx\""
Jul 23 11:27:01.651505 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.651457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vjptn"]
Jul 23 11:27:01.651640 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.651524    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5c96j"]
Jul 23 11:27:01.651640 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.651569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.654456 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.654431    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 11:27:01.654585 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.654486    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-89ttz\""
Jul 23 11:27:01.654771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.654755    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 11:27:01.654938 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.654925    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 11:27:01.710141 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710088    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710141 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710391 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710247    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710391 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdx9x\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710391 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd2n6\" (UniqueName: \"kubernetes.io/projected/5992ce67-f59b-4fbd-ab61-0708b33410df-kube-api-access-sd2n6\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.710573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5992ce67-f59b-4fbd-ab61-0708b33410df-tmp-dir\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.710573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710522    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.710573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710563    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5992ce67-f59b-4fbd-ab61-0708b33410df-config-volume\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.710765 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.710765 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.710602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.811904 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.811824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.811904 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.811860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812012    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd2n6\" (UniqueName: \"kubernetes.io/projected/5992ce67-f59b-4fbd-ab61-0708b33410df-kube-api-access-sd2n6\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.812168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5992ce67-f59b-4fbd-ab61-0708b33410df-tmp-dir\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.812168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlsfz\" (UniqueName: \"kubernetes.io/projected/5d62d78d-4f12-4db9-948f-f809d037f32f-kube-api-access-tlsfz\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.812168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5992ce67-f59b-4fbd-ab61-0708b33410df-config-volume\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812180    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d62d78d-4f12-4db9-948f-f809d037f32f-cert\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:01.812256    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:01.812270    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812274    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:01.812335    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.312314298 +0000 UTC m=+36.597780057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:01.812405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812379    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdx9x\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812889 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5992ce67-f59b-4fbd-ab61-0708b33410df-tmp-dir\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.812943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.812916    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.812994 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:01.812982    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:01.813044 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:01.813026    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.313013123 +0000 UTC m=+36.598478886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:01.813175 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.813155    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.813318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.813300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5992ce67-f59b-4fbd-ab61-0708b33410df-config-volume\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.813961 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.813938    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.817202 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.817174    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:01.817507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.817482    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.817629 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.817542    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.821900 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.821876    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdx9x\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.822271 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.822239    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:01.822354 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.822322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd2n6\" (UniqueName: \"kubernetes.io/projected/5992ce67-f59b-4fbd-ab61-0708b33410df-kube-api-access-sd2n6\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:01.910942 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.910879    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:01.912814 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.912791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlsfz\" (UniqueName: \"kubernetes.io/projected/5d62d78d-4f12-4db9-948f-f809d037f32f-kube-api-access-tlsfz\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.912911 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.912835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d62d78d-4f12-4db9-948f-f809d037f32f-cert\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.915591 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.915566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d62d78d-4f12-4db9-948f-f809d037f32f-cert\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.922413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.922389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlsfz\" (UniqueName: \"kubernetes.io/projected/5d62d78d-4f12-4db9-948f-f809d037f32f-kube-api-access-tlsfz\") pod \"ingress-canary-5c96j\" (UID: \"5d62d78d-4f12-4db9-948f-f809d037f32f\") " pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.941490 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.941440    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"]
Jul 23 11:27:01.959479 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.959428    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"]
Jul 23 11:27:01.959643 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.959545    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"
Jul 23 11:27:01.961110 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.961086    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5c96j"
Jul 23 11:27:01.963548 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.963524    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 23 11:27:01.963892 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.963872    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:01.965481 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.965436    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-b67nv\""
Jul 23 11:27:01.968740 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.967336    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"]
Jul 23 11:27:01.981311 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.981288    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"]
Jul 23 11:27:01.981489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.981448    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:01.988509 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.987449    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 11:27:01.988509 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.987537    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jmphl\""
Jul 23 11:27:01.988509 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.987599    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 23 11:27:01.988509 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.987449    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 11:27:01.991690 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.991669    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 23 11:27:01.996137 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.996113    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-g295d"]
Jul 23 11:27:01.996930 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:01.996910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.000198 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.000177    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 23 11:27:02.000319 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.000199    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.000838 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.000819    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8shz2\""
Jul 23 11:27:02.000940 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.000893    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 23 11:27:02.001039 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.001022    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.010422 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.010400    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"]
Jul 23 11:27:02.010422 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.010434    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"]
Jul 23 11:27:02.010611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.010509    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.013953 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.013934    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.014179 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.014161    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z5vr5\""
Jul 23 11:27:02.014525 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.014509    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.015093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.015064    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 23 11:27:02.015359 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.015327    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 23 11:27:02.017343 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.017323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-g295d"]
Jul 23 11:27:02.029366 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.029346    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 23 11:27:02.058937 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.058892    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"]
Jul 23 11:27:02.078136 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.077882    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4dp7h"]
Jul 23 11:27:02.078136 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.078441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.081402 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.081377    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 23 11:27:02.081556 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.081407    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 23 11:27:02.081556 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.081417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9sqq6\""
Jul 23 11:27:02.093935 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.093906    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"]
Jul 23 11:27:02.094055 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.094015    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.096905 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.096880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 23 11:27:02.097042 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.096947    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.097209 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.097189    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.097449 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.097430    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 23 11:27:02.097567 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.097492    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zb47v\""
Jul 23 11:27:02.106580 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.106554    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 23 11:27:02.111097 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.111076    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54c8b47ff6-g6ndr"]
Jul 23 11:27:02.111212 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.111195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.113778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwzp\" (UniqueName: \"kubernetes.io/projected/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-kube-api-access-cpwzp\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.113932 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113911    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.114013 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113942    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2bhjs\""
Jul 23 11:27:02.114013 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113967    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.114120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113783    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 23 11:27:02.114199 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.113922    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-tmp\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114298 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114298 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-serving-cert\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6567\" (UniqueName: \"kubernetes.io/projected/8781f6f3-451a-4393-a575-e16ac00e2687-kube-api-access-d6567\") pod \"volume-data-source-validator-78cf4c6f6-njk2f\" (UID: \"8781f6f3-451a-4393-a575-e16ac00e2687\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"
Jul 23 11:27:02.114401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea610113-71c0-45ed-b083-edebe6288113-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.114401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114354    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn7ch\" (UniqueName: \"kubernetes.io/projected/ea610113-71c0-45ed-b083-edebe6288113-kube-api-access-sn7ch\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.114401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbxt8\" (UniqueName: \"kubernetes.io/projected/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-kube-api-access-lbxt8\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114426    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.114611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114491    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea610113-71c0-45ed-b083-edebe6288113-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.114611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-service-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114590    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-snapshots\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.114765 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.114622    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.124555 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.124531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4dp7h"]
Jul 23 11:27:02.124690 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.124574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"]
Jul 23 11:27:02.124690 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.124687    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"]
Jul 23 11:27:02.124788 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.124693    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.124788 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.124714    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54c8b47ff6-g6ndr"]
Jul 23 11:27:02.127695 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.127666    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 23 11:27:02.128692 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128671    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 23 11:27:02.128692 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128687    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 23 11:27:02.128804 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128723    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.128804 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128773    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.128877 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128819    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-62s7d\""
Jul 23 11:27:02.128993 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.128979    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 23 11:27:02.142240 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.142215    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"]
Jul 23 11:27:02.154339 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.154309    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"]
Jul 23 11:27:02.154514 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.154437    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"
Jul 23 11:27:02.157548 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.157524    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.157548 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.157541    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rfvjk\""
Jul 23 11:27:02.157729 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.157573    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.172732 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.172706    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"]
Jul 23 11:27:02.172839 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.172744    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"]
Jul 23 11:27:02.172839 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.172803    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.176214 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.175815    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 23 11:27:02.176214 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.175991    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 23 11:27:02.176214 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.176080    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:02.176436 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.176221    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 23 11:27:02.176436 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.176303    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lw5xg\""
Jul 23 11:27:02.197121 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.197085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:02.215527 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.215715 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215541    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.215715 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-serving-cert\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.215715 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6a41630-31a9-4f8d-a321-be18df3b14e9-serving-cert\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.215715 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-trusted-ca\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.215921 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6567\" (UniqueName: \"kubernetes.io/projected/8781f6f3-451a-4393-a575-e16ac00e2687-kube-api-access-d6567\") pod \"volume-data-source-validator-78cf4c6f6-njk2f\" (UID: \"8781f6f3-451a-4393-a575-e16ac00e2687\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"
Jul 23 11:27:02.215921 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea610113-71c0-45ed-b083-edebe6288113-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.215921 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn7ch\" (UniqueName: \"kubernetes.io/projected/ea610113-71c0-45ed-b083-edebe6288113-kube-api-access-sn7ch\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.215921 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbxt8\" (UniqueName: \"kubernetes.io/projected/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-kube-api-access-lbxt8\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215960    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea610113-71c0-45ed-b083-edebe6288113-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.215998    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-stats-auth\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216029    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-config\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-service-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.216119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216116    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216141    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjvbl\" (UniqueName: \"kubernetes.io/projected/a6a41630-31a9-4f8d-a321-be18df3b14e9-kube-api-access-fjvbl\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-default-certificate\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f9n4\" (UniqueName: \"kubernetes.io/projected/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-kube-api-access-4f9n4\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216231    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-snapshots\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216324    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwzp\" (UniqueName: \"kubernetes.io/projected/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-kube-api-access-cpwzp\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.216407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216381    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt24z\" (UniqueName: \"kubernetes.io/projected/c1789000-4ac8-4f02-90d8-465dd0a33d2b-kube-api-access-jt24z\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-tmp\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216535    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ea610113-71c0-45ed-b083-edebe6288113-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216556    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.216679    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.216746    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.716728233 +0000 UTC m=+37.002193983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:02.216836 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216751    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-tmp\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.217161 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.216985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.217271 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.217254    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-service-ca-bundle\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.217381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.217360    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-snapshots\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.218522 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.218501    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-serving-cert\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.219052 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.219033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea610113-71c0-45ed-b083-edebe6288113-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.227671 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.227652    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbxt8\" (UniqueName: \"kubernetes.io/projected/90b4c78f-dfd7-4365-8510-c0bf3a7acac8-kube-api-access-lbxt8\") pod \"insights-operator-555d9d7885-g295d\" (UID: \"90b4c78f-dfd7-4365-8510-c0bf3a7acac8\") " pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.227920 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.227898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwzp\" (UniqueName: \"kubernetes.io/projected/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-kube-api-access-cpwzp\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.228198 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.228177    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6567\" (UniqueName: \"kubernetes.io/projected/8781f6f3-451a-4393-a575-e16ac00e2687-kube-api-access-d6567\") pod \"volume-data-source-validator-78cf4c6f6-njk2f\" (UID: \"8781f6f3-451a-4393-a575-e16ac00e2687\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"
Jul 23 11:27:02.228829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.228811    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn7ch\" (UniqueName: \"kubernetes.io/projected/ea610113-71c0-45ed-b083-edebe6288113-kube-api-access-sn7ch\") pod \"kube-storage-version-migrator-operator-7658d9b749-zzq4x\" (UID: \"ea610113-71c0-45ed-b083-edebe6288113\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.272003 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.271965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"
Jul 23 11:27:02.292224 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.292200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:27:02.292224 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.292218    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:27:02.292442 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.292428    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:02.294991 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.294888    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 11:27:02.295106 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.294999    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 11:27:02.295106 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.295019    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zh6tm\""
Jul 23 11:27:02.295217 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.295174    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8j6tk\""
Jul 23 11:27:02.308010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.307987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"
Jul 23 11:27:02.317752 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:02.317882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.317882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjvbl\" (UniqueName: \"kubernetes.io/projected/a6a41630-31a9-4f8d-a321-be18df3b14e9-kube-api-access-fjvbl\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.317882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-default-certificate\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.317882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f9n4\" (UniqueName: \"kubernetes.io/projected/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-kube-api-access-4f9n4\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.317882 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.317875    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.317936    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.317920238 +0000 UTC m=+37.603385982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.317974    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318025    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.818008794 +0000 UTC m=+37.103474540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.317877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.318126 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318117    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c9xl\" (UniqueName: \"kubernetes.io/projected/aeb25ac3-55d3-4a44-b0db-2924d911b697-kube-api-access-8c9xl\") pod \"network-check-source-7bc5b6bf85-vljg8\" (UID: \"aeb25ac3-55d3-4a44-b0db-2924d911b697\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt24z\" (UniqueName: \"kubernetes.io/projected/c1789000-4ac8-4f02-90d8-465dd0a33d2b-kube-api-access-jt24z\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318201    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318253    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls podName:c1789000-4ac8-4f02-90d8-465dd0a33d2b nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.818240082 +0000 UTC m=+37.103705830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tlxbb" (UID: "c1789000-4ac8-4f02-90d8-465dd0a33d2b") : secret "samples-operator-tls" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318296    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318308    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318339    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.318328722 +0000 UTC m=+37.603794473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318356    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318396    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.818383138 +0000 UTC m=+37.103848891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6a41630-31a9-4f8d-a321-be18df3b14e9-serving-cert\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.318424 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.318414    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:02.818405155 +0000 UTC m=+37.103870915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-trusted-ca\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbd7\" (UniqueName: \"kubernetes.io/projected/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-kube-api-access-7wbd7\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318609    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-config\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-stats-auth\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.319190 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.318670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-config\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.319530 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.319260    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-trusted-ca\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.319530 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.319298    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a6a41630-31a9-4f8d-a321-be18df3b14e9-config\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.320977 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.320959    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-g295d"
Jul 23 11:27:02.321073 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.320975    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a6a41630-31a9-4f8d-a321-be18df3b14e9-serving-cert\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.321368 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.321340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-default-certificate\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.321415 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.321387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-stats-auth\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.327778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.327683    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:02.328187 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.328129    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f9n4\" (UniqueName: \"kubernetes.io/projected/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-kube-api-access-4f9n4\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.328351 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.328170    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt24z\" (UniqueName: \"kubernetes.io/projected/c1789000-4ac8-4f02-90d8-465dd0a33d2b-kube-api-access-jt24z\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.328351 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.328326    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjvbl\" (UniqueName: \"kubernetes.io/projected/a6a41630-31a9-4f8d-a321-be18df3b14e9-kube-api-access-fjvbl\") pod \"console-operator-565b7b569d-4dp7h\" (UID: \"a6a41630-31a9-4f8d-a321-be18df3b14e9\") " pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.406211 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.406168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:02.420191 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.420163    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.420307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.420230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbd7\" (UniqueName: \"kubernetes.io/projected/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-kube-api-access-7wbd7\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.420307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.420287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-config\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.420418 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.420401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c9xl\" (UniqueName: \"kubernetes.io/projected/aeb25ac3-55d3-4a44-b0db-2924d911b697-kube-api-access-8c9xl\") pod \"network-check-source-7bc5b6bf85-vljg8\" (UID: \"aeb25ac3-55d3-4a44-b0db-2924d911b697\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"
Jul 23 11:27:02.421072 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.421050    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-config\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.422777 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.422759    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.430594 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.430564    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbd7\" (UniqueName: \"kubernetes.io/projected/b0f4ac95-4fd6-4e47-8327-c7c6bf843339-kube-api-access-7wbd7\") pod \"service-ca-operator-5bcbc94c78-jdb9p\" (UID: \"b0f4ac95-4fd6-4e47-8327-c7c6bf843339\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.432915 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.431116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c9xl\" (UniqueName: \"kubernetes.io/projected/aeb25ac3-55d3-4a44-b0db-2924d911b697-kube-api-access-8c9xl\") pod \"network-check-source-7bc5b6bf85-vljg8\" (UID: \"aeb25ac3-55d3-4a44-b0db-2924d911b697\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"
Jul 23 11:27:02.465396 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.465358    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"
Jul 23 11:27:02.482361 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.482325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"
Jul 23 11:27:02.723220 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.723174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:02.723910 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.723376    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:02.723910 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.723542    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.723519195 +0000 UTC m=+38.008984949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:02.733318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.733286    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:02.824187 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.824151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.824345 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.824276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:02.824345 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.824310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:02.824419 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824354    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.824330091 +0000 UTC m=+38.109795838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:02.824419 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:02.824391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:02.824419 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824402    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 11:27:02.824419 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824403    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:02.824630 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824450    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls podName:c1789000-4ac8-4f02-90d8-465dd0a33d2b nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.824435649 +0000 UTC m=+38.109901393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tlxbb" (UID: "c1789000-4ac8-4f02-90d8-465dd0a33d2b") : secret "samples-operator-tls" not found
Jul 23 11:27:02.824630 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824482    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.824474871 +0000 UTC m=+38.109940628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:02.824630 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824520    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:02.824630 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:02.824564    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:03.824551865 +0000 UTC m=+38.110017612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.331059    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.331424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.331638    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.331656    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.331715    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.331697168 +0000 UTC m=+39.617162931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.332179    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:03.332278 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.332234    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.332215494 +0000 UTC m=+39.617681239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:03.443094 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.443066    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x"]
Jul 23 11:27:03.446526 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.446496    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5c96j"]
Jul 23 11:27:03.457910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.457882    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-g295d"]
Jul 23 11:27:03.460621 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.460513    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f"]
Jul 23 11:27:03.461388 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.461317    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p"]
Jul 23 11:27:03.464942 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.464921    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-4dp7h"]
Jul 23 11:27:03.472723 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.472694    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea610113_71c0_45ed_b083_edebe6288113.slice/crio-ce9dda150005b6cd50fbabfbec9747a5d6796ee604adf94d917931c5d5487053 WatchSource:0}: Error finding container ce9dda150005b6cd50fbabfbec9747a5d6796ee604adf94d917931c5d5487053: Status 404 returned error can't find the container with id ce9dda150005b6cd50fbabfbec9747a5d6796ee604adf94d917931c5d5487053
Jul 23 11:27:03.474177 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.474150    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d62d78d_4f12_4db9_948f_f809d037f32f.slice/crio-2063c6b648f4a26d2d5e7c54d8be515e20e646e9fdaf8b17c6dd22ebf0a21f79 WatchSource:0}: Error finding container 2063c6b648f4a26d2d5e7c54d8be515e20e646e9fdaf8b17c6dd22ebf0a21f79: Status 404 returned error can't find the container with id 2063c6b648f4a26d2d5e7c54d8be515e20e646e9fdaf8b17c6dd22ebf0a21f79
Jul 23 11:27:03.474499 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.474395    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8"]
Jul 23 11:27:03.474975 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.474942    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b4c78f_dfd7_4365_8510_c0bf3a7acac8.slice/crio-386ab68f33dbbf39c75f8aaa906211e72680a22c929dca0edc1a8a6d4a2aa9fb WatchSource:0}: Error finding container 386ab68f33dbbf39c75f8aaa906211e72680a22c929dca0edc1a8a6d4a2aa9fb: Status 404 returned error can't find the container with id 386ab68f33dbbf39c75f8aaa906211e72680a22c929dca0edc1a8a6d4a2aa9fb
Jul 23 11:27:03.476551 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.476088    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8781f6f3_451a_4393_a575_e16ac00e2687.slice/crio-64d75ddea474434a8f83e0f1c3327c116de0e076ee7839c97458f10c53ac3f5d WatchSource:0}: Error finding container 64d75ddea474434a8f83e0f1c3327c116de0e076ee7839c97458f10c53ac3f5d: Status 404 returned error can't find the container with id 64d75ddea474434a8f83e0f1c3327c116de0e076ee7839c97458f10c53ac3f5d
Jul 23 11:27:03.478981 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.478809    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a41630_31a9_4f8d_a321_be18df3b14e9.slice/crio-d761ee89d5cfa0ba1ec4aaee017d2ab3bc4f1aaf650aa6dfb3e4b190590f0428 WatchSource:0}: Error finding container d761ee89d5cfa0ba1ec4aaee017d2ab3bc4f1aaf650aa6dfb3e4b190590f0428: Status 404 returned error can't find the container with id d761ee89d5cfa0ba1ec4aaee017d2ab3bc4f1aaf650aa6dfb3e4b190590f0428
Jul 23 11:27:03.479456 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:03.479373    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb25ac3_55d3_4a44_b0db_2924d911b697.slice/crio-a0759e03891499d41fc7d24e8ef5f51e2bfc5efc466342e562149b834d13f021 WatchSource:0}: Error finding container a0759e03891499d41fc7d24e8ef5f51e2bfc5efc466342e562149b834d13f021: Status 404 returned error can't find the container with id a0759e03891499d41fc7d24e8ef5f51e2bfc5efc466342e562149b834d13f021
Jul 23 11:27:03.546207 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.546174    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" event={"ID":"a6a41630-31a9-4f8d-a321-be18df3b14e9","Type":"ContainerStarted","Data":"d761ee89d5cfa0ba1ec4aaee017d2ab3bc4f1aaf650aa6dfb3e4b190590f0428"}
Jul 23 11:27:03.547317 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.547294    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f" event={"ID":"8781f6f3-451a-4393-a575-e16ac00e2687","Type":"ContainerStarted","Data":"64d75ddea474434a8f83e0f1c3327c116de0e076ee7839c97458f10c53ac3f5d"}
Jul 23 11:27:03.548425 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.548402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8" event={"ID":"aeb25ac3-55d3-4a44-b0db-2924d911b697","Type":"ContainerStarted","Data":"a0759e03891499d41fc7d24e8ef5f51e2bfc5efc466342e562149b834d13f021"}
Jul 23 11:27:03.549523 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.549489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x" event={"ID":"ea610113-71c0-45ed-b083-edebe6288113","Type":"ContainerStarted","Data":"ce9dda150005b6cd50fbabfbec9747a5d6796ee604adf94d917931c5d5487053"}
Jul 23 11:27:03.550477 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.550442    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p" event={"ID":"b0f4ac95-4fd6-4e47-8327-c7c6bf843339","Type":"ContainerStarted","Data":"25f8cb6ab2bab8dc8b410d2a3e2abaad9a2086d6dc8bee819aa7b19abdb59646"}
Jul 23 11:27:03.551447 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.551424    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5c96j" event={"ID":"5d62d78d-4f12-4db9-948f-f809d037f32f","Type":"ContainerStarted","Data":"2063c6b648f4a26d2d5e7c54d8be515e20e646e9fdaf8b17c6dd22ebf0a21f79"}
Jul 23 11:27:03.552507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.552487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-g295d" event={"ID":"90b4c78f-dfd7-4365-8510-c0bf3a7acac8","Type":"ContainerStarted","Data":"386ab68f33dbbf39c75f8aaa906211e72680a22c929dca0edc1a8a6d4a2aa9fb"}
Jul 23 11:27:03.735814 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.735778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:03.736189 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.735897    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:03.736189 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.735951    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.735937239 +0000 UTC m=+40.021402984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:03.836627 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.836585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:03.836627 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.836643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836731    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836737    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836781    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls podName:c1789000-4ac8-4f02-90d8-465dd0a33d2b nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.836767205 +0000 UTC m=+40.122232949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tlxbb" (UID: "c1789000-4ac8-4f02-90d8-465dd0a33d2b") : secret "samples-operator-tls" not found
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.836778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836794    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.83678809 +0000 UTC m=+40.122253834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:03.836848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:03.836898 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836897    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:03.837152 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836951    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.836937315 +0000 UTC m=+40.122403069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:03.837152 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:03.836973    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:05.836958593 +0000 UTC m=+40.122424340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:04.566525 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:04.566485    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="8c8c3b4f84d0066ebe269030d9a63dbc8425a2846adc115731fa862584ab6370" exitCode=0
Jul 23 11:27:04.566791 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:04.566583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"8c8c3b4f84d0066ebe269030d9a63dbc8425a2846adc115731fa862584ab6370"}
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.363343    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.363721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.363947    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.363968    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.364033    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.364013171 +0000 UTC m=+43.649478935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.364516    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:05.364648 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.364570    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.364553822 +0000 UTC m=+43.650019585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:05.580794 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.580637    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1086489-8fd3-4e04-a8ee-290c842353d7" containerID="d7ffe62a00cc560a5dc4fdf21bf1ad91965a994ae9b6c8c6912fcdb3f6c4f348" exitCode=0
Jul 23 11:27:05.580794 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.580696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerDied","Data":"d7ffe62a00cc560a5dc4fdf21bf1ad91965a994ae9b6c8c6912fcdb3f6c4f348"}
Jul 23 11:27:05.774320 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.772917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:05.774320 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.773130    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:05.774320 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.773202    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.773182952 +0000 UTC m=+44.058648700 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:05.873986 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.873948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:05.874182 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.874018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:05.874182 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.874058    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:05.874182 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:05.874105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:05.874367 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874301    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.874281082 +0000 UTC m=+44.159746851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:05.874735 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874603    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:05.874735 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874625    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:05.874735 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874673    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.874654103 +0000 UTC m=+44.160119850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:05.874735 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874692    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.874682732 +0000 UTC m=+44.160148482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:05.874735 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874723    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 11:27:05.875076 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:05.874778    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls podName:c1789000-4ac8-4f02-90d8-465dd0a33d2b nodeName:}" failed. No retries permitted until 2026-07-23 11:27:09.874760696 +0000 UTC m=+44.160226446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tlxbb" (UID: "c1789000-4ac8-4f02-90d8-465dd0a33d2b") : secret "samples-operator-tls" not found
Jul 23 11:27:06.832560 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:06.832518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:06.888418 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:06.888380    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log" limit=1024
Jul 23 11:27:09.398062 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.398017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.398145    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.398196    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.398277    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.398253381 +0000 UTC m=+51.683719126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.398319    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.398343    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:09.398522 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.398405    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.398385568 +0000 UTC m=+51.683851324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:09.802163 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.802060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:09.802347 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.802251    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:09.802347 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.802325    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.802305131 +0000 UTC m=+52.087770879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:09.903212 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.903165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:09.903390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.903222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:09.903390 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903324    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 23 11:27:09.903390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.903348    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:09.903390 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903324    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:09.903390 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903387    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls podName:c1789000-4ac8-4f02-90d8-465dd0a33d2b nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.903368127 +0000 UTC m=+52.188833883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tlxbb" (UID: "c1789000-4ac8-4f02-90d8-465dd0a33d2b") : secret "samples-operator-tls" not found
Jul 23 11:27:09.903679 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903407    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.903396453 +0000 UTC m=+52.188862196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:09.903679 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903423    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:09.903679 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:09.903451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:09.903679 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903548    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.903458639 +0000 UTC m=+52.188924383 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:09.903679 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:09.903570    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:17.903562847 +0000 UTC m=+52.189028591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:11.598895 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.598816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p" event={"ID":"b0f4ac95-4fd6-4e47-8327-c7c6bf843339","Type":"ContainerStarted","Data":"8193b723a3af662718582014459414f6f78ced9281e9de6b8ba4a66708ce21e3"}
Jul 23 11:27:11.603750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.603717    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" event={"ID":"a1086489-8fd3-4e04-a8ee-290c842353d7","Type":"ContainerStarted","Data":"887de1f3136f52bc474f2856fd4f312b973cad74f474060b741b7b5063cb958a"}
Jul 23 11:27:11.605224 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.605193    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f" event={"ID":"8781f6f3-451a-4393-a575-e16ac00e2687","Type":"ContainerStarted","Data":"6a201da0f9d66e9e0454d33f49bfbbae82a29d52c8d3c260c62b5623fc81a3bd"}
Jul 23 11:27:11.615959 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.615911    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p" podStartSLOduration=1.702035345 podStartE2EDuration="9.61589411s" totalImagesPullingTime="7.913858765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:11.615313027 +0000 UTC m=+45.900778792" watchObservedRunningTime="2026-07-23 11:27:11.61589411 +0000 UTC m=+45.901359873"
Jul 23 11:27:11.640658 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.640598    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bsjcb" podStartSLOduration=14.934984191 podStartE2EDuration="45.64057577s" totalImagesPullingTime="30.705591579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:11.639342544 +0000 UTC m=+45.924808307" watchObservedRunningTime="2026-07-23 11:27:11.64057577 +0000 UTC m=+45.926041570"
Jul 23 11:27:11.657108 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:11.656627    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-njk2f" podStartSLOduration=2.744941919 podStartE2EDuration="10.656608051s" totalImagesPullingTime="7.911666132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:11.654947462 +0000 UTC m=+45.940413228" watchObservedRunningTime="2026-07-23 11:27:11.656608051 +0000 UTC m=+45.942073818"
Jul 23 11:27:12.611196 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.611153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x" event={"ID":"ea610113-71c0-45ed-b083-edebe6288113","Type":"ContainerStarted","Data":"85f001eb6076ba78b2323b0768c84e992f138eaae5743df7a8d07ec6da79d73a"}
Jul 23 11:27:12.612951 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.612883    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5c96j" event={"ID":"5d62d78d-4f12-4db9-948f-f809d037f32f","Type":"ContainerStarted","Data":"163afdfc53dfce6f8aac52c2b38fd59478f4cae5109a6bde905bf16e6b36c3cf"}
Jul 23 11:27:12.614606 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.614550    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-g295d" event={"ID":"90b4c78f-dfd7-4365-8510-c0bf3a7acac8","Type":"ContainerStarted","Data":"9161a167f17fe0ac532242c2479dc8a5d5b63b5ae5905365d9f39119a889ead6"}
Jul 23 11:27:12.616525 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.616507    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/0.log"
Jul 23 11:27:12.616638 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.616542    2574 generic.go:356] "Generic (PLEG): container finished" podID="a6a41630-31a9-4f8d-a321-be18df3b14e9" containerID="a86345893ea94114ceac3f6bec01919b656b19d0b3be9ada2d7aa0347679a1b6" exitCode=255
Jul 23 11:27:12.616638 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.616626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" event={"ID":"a6a41630-31a9-4f8d-a321-be18df3b14e9","Type":"ContainerDied","Data":"a86345893ea94114ceac3f6bec01919b656b19d0b3be9ada2d7aa0347679a1b6"}
Jul 23 11:27:12.616851 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.616836    2574 scope.go:117] "RemoveContainer" containerID="a86345893ea94114ceac3f6bec01919b656b19d0b3be9ada2d7aa0347679a1b6"
Jul 23 11:27:12.618713 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.618283    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8" event={"ID":"aeb25ac3-55d3-4a44-b0db-2924d911b697","Type":"ContainerStarted","Data":"582bbc1c6a5ce7d071c36f21bf2e96e3683f4a896a8c83918ee79a79d7511078"}
Jul 23 11:27:12.631125 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.631075    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x" podStartSLOduration=3.686790293 podStartE2EDuration="11.631055023s" totalImagesPullingTime="7.94426473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:12.629925822 +0000 UTC m=+46.915391582" watchObservedRunningTime="2026-07-23 11:27:12.631055023 +0000 UTC m=+46.916520790"
Jul 23 11:27:12.648378 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.648320    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-vljg8" podStartSLOduration=2.474679547 podStartE2EDuration="10.648299492s" totalImagesPullingTime="8.173619945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:12.646614272 +0000 UTC m=+46.932080063" watchObservedRunningTime="2026-07-23 11:27:12.648299492 +0000 UTC m=+46.933765259"
Jul 23 11:27:12.668093 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.668036    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-g295d" podStartSLOduration=3.754437023 podStartE2EDuration="11.668014986s" totalImagesPullingTime="7.913577963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:12.666967882 +0000 UTC m=+46.952433648" watchObservedRunningTime="2026-07-23 11:27:12.668014986 +0000 UTC m=+46.953480754"
Jul 23 11:27:12.715971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:12.715906    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5c96j" podStartSLOduration=3.772680881 podStartE2EDuration="11.715882699s" totalImagesPullingTime="7.943201818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:12.688857001 +0000 UTC m=+46.974322769" watchObservedRunningTime="2026-07-23 11:27:12.715882699 +0000 UTC m=+47.001348466"
Jul 23 11:27:13.138264 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.138204    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q4hbv"]
Jul 23 11:27:13.155892 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.155861    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q4hbv"]
Jul 23 11:27:13.156070 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.155994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.158827 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.158785    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfh54\""
Jul 23 11:27:13.158978 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.158824    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 11:27:13.159052 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.159002    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 11:27:13.233638 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.233602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.233813 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.233671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1adbad02-1655-44ea-b057-5629f0df8c47-crio-socket\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.233813 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.233775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.233902 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.233830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pgdb\" (UniqueName: \"kubernetes.io/projected/1adbad02-1655-44ea-b057-5629f0df8c47-kube-api-access-8pgdb\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.233902 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.233855    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1adbad02-1655-44ea-b057-5629f0df8c47-data-volume\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335199 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pgdb\" (UniqueName: \"kubernetes.io/projected/1adbad02-1655-44ea-b057-5629f0df8c47-kube-api-access-8pgdb\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335199 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335201    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1adbad02-1655-44ea-b057-5629f0df8c47-data-volume\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335440 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335246    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335440 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1adbad02-1655-44ea-b057-5629f0df8c47-crio-socket\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335440 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335440 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.335406    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 11:27:13.335440 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.335440    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:13.335705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335411    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1adbad02-1655-44ea-b057-5629f0df8c47-crio-socket\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.335705 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.335495    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:13.835453052 +0000 UTC m=+48.120918796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : configmap "kube-rbac-proxy" not found
Jul 23 11:27:13.335705 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.335517    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:13.835506695 +0000 UTC m=+48.120972438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:13.335705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.335613    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1adbad02-1655-44ea-b057-5629f0df8c47-data-volume\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.346041 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.346014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pgdb\" (UniqueName: \"kubernetes.io/projected/1adbad02-1655-44ea-b057-5629f0df8c47-kube-api-access-8pgdb\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.486114 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.486039    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"]
Jul 23 11:27:13.502651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.502621    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"]
Jul 23 11:27:13.502888 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.502722    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"
Jul 23 11:27:13.506865 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.506835    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 23 11:27:13.506865 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.506843    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tnb72\""
Jul 23 11:27:13.507083 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.506902    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 23 11:27:13.536993 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.536958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bcqr\" (UniqueName: \"kubernetes.io/projected/48f361ec-0e22-4640-820f-0eb38d1b956c-kube-api-access-9bcqr\") pod \"migrator-7cc9c7ffd7-85b2p\" (UID: \"48f361ec-0e22-4640-820f-0eb38d1b956c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"
Jul 23 11:27:13.626379 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.626345    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:27:13.626824 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.626803    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/0.log"
Jul 23 11:27:13.626882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.626844    2574 generic.go:356] "Generic (PLEG): container finished" podID="a6a41630-31a9-4f8d-a321-be18df3b14e9" containerID="98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4" exitCode=255
Jul 23 11:27:13.627036 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.627001    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" event={"ID":"a6a41630-31a9-4f8d-a321-be18df3b14e9","Type":"ContainerDied","Data":"98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4"}
Jul 23 11:27:13.627159 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.627061    2574 scope.go:117] "RemoveContainer" containerID="a86345893ea94114ceac3f6bec01919b656b19d0b3be9ada2d7aa0347679a1b6"
Jul 23 11:27:13.627209 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.627184    2574 scope.go:117] "RemoveContainer" containerID="98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4"
Jul 23 11:27:13.627392 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.627364    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4dp7h_openshift-console-operator(a6a41630-31a9-4f8d-a321-be18df3b14e9)\"" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" podUID="a6a41630-31a9-4f8d-a321-be18df3b14e9"
Jul 23 11:27:13.637746 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.637711    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bcqr\" (UniqueName: \"kubernetes.io/projected/48f361ec-0e22-4640-820f-0eb38d1b956c-kube-api-access-9bcqr\") pod \"migrator-7cc9c7ffd7-85b2p\" (UID: \"48f361ec-0e22-4640-820f-0eb38d1b956c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"
Jul 23 11:27:13.647924 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.647896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bcqr\" (UniqueName: \"kubernetes.io/projected/48f361ec-0e22-4640-820f-0eb38d1b956c-kube-api-access-9bcqr\") pod \"migrator-7cc9c7ffd7-85b2p\" (UID: \"48f361ec-0e22-4640-820f-0eb38d1b956c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"
Jul 23 11:27:13.814204 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.814102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"
Jul 23 11:27:13.840626 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.840593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.840809 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.840707    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 11:27:13.840809 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.840712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:13.840809 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.840792    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:13.840809 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.840800    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:14.840778473 +0000 UTC m=+49.126244227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : configmap "kube-rbac-proxy" not found
Jul 23 11:27:13.841030 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:13.840833    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:14.840821393 +0000 UTC m=+49.126287140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:13.968238 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:13.968199    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p"]
Jul 23 11:27:13.971141 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:13.971113    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f361ec_0e22_4640_820f_0eb38d1b956c.slice/crio-f92b80e52e7785620c8a85b06a6376badf41df03a915328fed593ad0f4dbfa34 WatchSource:0}: Error finding container f92b80e52e7785620c8a85b06a6376badf41df03a915328fed593ad0f4dbfa34: Status 404 returned error can't find the container with id f92b80e52e7785620c8a85b06a6376badf41df03a915328fed593ad0f4dbfa34
Jul 23 11:27:14.631744 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.631718    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kbssr_9f2feb0a-1fb1-4137-abcb-64b3df097c96/dns-node-resolver/0.log"
Jul 23 11:27:14.632158 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.631957    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:27:14.632334 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.632316    2574 scope.go:117] "RemoveContainer" containerID="98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4"
Jul 23 11:27:14.632558 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:14.632537    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4dp7h_openshift-console-operator(a6a41630-31a9-4f8d-a321-be18df3b14e9)\"" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" podUID="a6a41630-31a9-4f8d-a321-be18df3b14e9"
Jul 23 11:27:14.633327 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.633306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p" event={"ID":"48f361ec-0e22-4640-820f-0eb38d1b956c","Type":"ContainerStarted","Data":"f92b80e52e7785620c8a85b06a6376badf41df03a915328fed593ad0f4dbfa34"}
Jul 23 11:27:14.850634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.850590    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:14.850823 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:14.850707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:14.850823 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:14.850728    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 11:27:14.850823 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:14.850813    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:16.850788434 +0000 UTC m=+51.136254182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : configmap "kube-rbac-proxy" not found
Jul 23 11:27:14.850823 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:14.850815    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:14.851005 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:14.850872    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:16.850860776 +0000 UTC m=+51.136326524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:15.431324 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.431260    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pg9xf_64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25/node-ca/0.log"
Jul 23 11:27:15.755342 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.755313    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ppzd5"]
Jul 23 11:27:15.776656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.776615    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ppzd5"]
Jul 23 11:27:15.776656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.776642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.779737 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.779715    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 23 11:27:15.779851 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.779745    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 23 11:27:15.779851 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.779751    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 23 11:27:15.779851 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.779813    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 23 11:27:15.781250 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.781236    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vzv2w\""
Jul 23 11:27:15.860610 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.860581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp4gr\" (UniqueName: \"kubernetes.io/projected/6f9d44d2-22eb-4997-ad40-2730f48f6294-kube-api-access-vp4gr\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.860780 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.860644    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-key\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.860780 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.860746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-cabundle\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.961998 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.961964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp4gr\" (UniqueName: \"kubernetes.io/projected/6f9d44d2-22eb-4997-ad40-2730f48f6294-kube-api-access-vp4gr\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.962142 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.962060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-key\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.962142 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.962092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-cabundle\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.962970 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.962937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-cabundle\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.964492 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.964454    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6f9d44d2-22eb-4997-ad40-2730f48f6294-signing-key\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:15.971387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:15.971364    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp4gr\" (UniqueName: \"kubernetes.io/projected/6f9d44d2-22eb-4997-ad40-2730f48f6294-kube-api-access-vp4gr\") pod \"service-ca-58598695f7-ppzd5\" (UID: \"6f9d44d2-22eb-4997-ad40-2730f48f6294\") " pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:16.031876 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.031796    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5c96j_5d62d78d-4f12-4db9-948f-f809d037f32f/serve-healthcheck-canary/0.log"
Jul 23 11:27:16.086592 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.086561    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ppzd5"
Jul 23 11:27:16.228415 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.228372    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ppzd5"]
Jul 23 11:27:16.231041 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:16.231019    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9d44d2_22eb_4997_ad40_2730f48f6294.slice/crio-4669517195ff79af671991fc6c5592d36a678a3953c2af2ae2cbf5bff790340b WatchSource:0}: Error finding container 4669517195ff79af671991fc6c5592d36a678a3953c2af2ae2cbf5bff790340b: Status 404 returned error can't find the container with id 4669517195ff79af671991fc6c5592d36a678a3953c2af2ae2cbf5bff790340b
Jul 23 11:27:16.641783 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.641749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p" event={"ID":"48f361ec-0e22-4640-820f-0eb38d1b956c","Type":"ContainerStarted","Data":"d98e07f6e6fc091e3c3635451d808a72bb69367ab5a9f2b7ab674de702ccd44e"}
Jul 23 11:27:16.641962 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.641806    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p" event={"ID":"48f361ec-0e22-4640-820f-0eb38d1b956c","Type":"ContainerStarted","Data":"681502913aa412722357735cb76c025c56fef68cdf8c00c4952ca1afd3f6ecb6"}
Jul 23 11:27:16.643322 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.643294    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ppzd5" event={"ID":"6f9d44d2-22eb-4997-ad40-2730f48f6294","Type":"ContainerStarted","Data":"c6c632dd445eea6b8e35280f155dc09ecae736d104fc3dd6fa55d0b94abdefba"}
Jul 23 11:27:16.643436 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.643338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ppzd5" event={"ID":"6f9d44d2-22eb-4997-ad40-2730f48f6294","Type":"ContainerStarted","Data":"4669517195ff79af671991fc6c5592d36a678a3953c2af2ae2cbf5bff790340b"}
Jul 23 11:27:16.661282 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.661219    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-85b2p" podStartSLOduration=1.8813821210000001 podStartE2EDuration="3.661197644s" totalImagesPullingTime="1.779815523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:16.659758723 +0000 UTC m=+50.945224488" watchObservedRunningTime="2026-07-23 11:27:16.661197644 +0000 UTC m=+50.946663411"
Jul 23 11:27:16.678061 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.678005    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-ppzd5" podStartSLOduration=1.677985098 podStartE2EDuration="1.677985098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:16.677183443 +0000 UTC m=+50.962649209" watchObservedRunningTime="2026-07-23 11:27:16.677985098 +0000 UTC m=+50.963450866"
Jul 23 11:27:16.872164 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.872125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:16.872583 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:16.872233    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:16.872583 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:16.872387    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:16.872583 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:16.872435    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:20.872421271 +0000 UTC m=+55.157887015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:16.872697 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:16.872583    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 11:27:16.872697 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:16.872655    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:20.872635985 +0000 UTC m=+55.158101733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : configmap "kube-rbac-proxy" not found
Jul 23 11:27:17.478549 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.478511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:17.478745 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.478641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:17.478745 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.478688    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 11:27:17.478745 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.478744    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 11:27:17.478918 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.478751    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls podName:5992ce67-f59b-4fbd-ab61-0708b33410df nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.478733455 +0000 UTC m=+67.764199216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls") pod "dns-default-vjptn" (UID: "5992ce67-f59b-4fbd-ab61-0708b33410df") : secret "dns-default-metrics-tls" not found
Jul 23 11:27:17.478918 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.478755    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56d49f9bf-ftdc7: secret "image-registry-tls" not found
Jul 23 11:27:17.478918 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.478788    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls podName:0c36f467-882d-4da8-87f8-c9414a008ad7 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.478778886 +0000 UTC m=+67.764244629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls") pod "image-registry-56d49f9bf-ftdc7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7") : secret "image-registry-tls" not found
Jul 23 11:27:17.882867 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.882831    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:17.883271 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.882985    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:17.883271 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.883053    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls podName:9d4e3a23-07a2-4660-9ec8-a1a43324d0f0 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.883038239 +0000 UTC m=+68.168503984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-dgxv2" (UID: "9d4e3a23-07a2-4660-9ec8-a1a43324d0f0") : secret "cluster-monitoring-operator-tls" not found
Jul 23 11:27:17.984017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.983978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:17.984017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.984023    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.984051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.984092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.984162    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.984208    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.984245    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.984226772 +0000 UTC m=+68.269692518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : configmap references non-existent config key: service-ca.crt
Jul 23 11:27:17.984263 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.984264    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert podName:0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.984253465 +0000 UTC m=+68.269719210 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xnm87" (UID: "0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a") : secret "networking-console-plugin-cert" not found
Jul 23 11:27:17.984454 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:17.984278    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs podName:702a7933-1e2c-4e40-9bc5-f0ed5ee0e139 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:33.984272301 +0000 UTC m=+68.269738046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs") pod "router-default-54c8b47ff6-g6ndr" (UID: "702a7933-1e2c-4e40-9bc5-f0ed5ee0e139") : secret "router-metrics-certs-default" not found
Jul 23 11:27:17.986662 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:17.986636    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1789000-4ac8-4f02-90d8-465dd0a33d2b-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tlxbb\" (UID: \"c1789000-4ac8-4f02-90d8-465dd0a33d2b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:18.020814 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:18.020725    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"
Jul 23 11:27:18.172848 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:18.172767    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb"]
Jul 23 11:27:18.651890 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:18.651856    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb" event={"ID":"c1789000-4ac8-4f02-90d8-465dd0a33d2b","Type":"ContainerStarted","Data":"01200353ebe7e2fab409afdb36aa71f6a901852c28eab3bb210b388757ab348e"}
Jul 23 11:27:20.660566 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.660528    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb" event={"ID":"c1789000-4ac8-4f02-90d8-465dd0a33d2b","Type":"ContainerStarted","Data":"5e4027bbae372a2d36cb9f3535f296973c57e834528be87a86e254997da6e895"}
Jul 23 11:27:20.660925 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.660593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb" event={"ID":"c1789000-4ac8-4f02-90d8-465dd0a33d2b","Type":"ContainerStarted","Data":"56e937af95cbfc8f32b2b8914463c3eb3b325699d846f66edfdd249fb2cfe9a9"}
Jul 23 11:27:20.679566 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.679517    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tlxbb" podStartSLOduration=16.923949951 podStartE2EDuration="18.679501935s" totalImagesPullingTime="1.755551984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:20.679240477 +0000 UTC m=+54.964706245" watchObservedRunningTime="2026-07-23 11:27:20.679501935 +0000 UTC m=+54.964967700"
Jul 23 11:27:20.913160 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.913080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:20.913306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.913184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:20.913306 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:20.913303    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:20.913368 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:20.913363    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls podName:1adbad02-1655-44ea-b057-5629f0df8c47 nodeName:}" failed. No retries permitted until 2026-07-23 11:27:28.913346079 +0000 UTC m=+63.198811832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q4hbv" (UID: "1adbad02-1655-44ea-b057-5629f0df8c47") : secret "insights-runtime-extractor-tls" not found
Jul 23 11:27:20.913642 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:20.913622    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1adbad02-1655-44ea-b057-5629f0df8c47-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:22.406569 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:22.406525    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:22.407072 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:22.406582    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:22.407072 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:22.406933    2574 scope.go:117] "RemoveContainer" containerID="98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4"
Jul 23 11:27:22.407155 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:27:22.407083    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-4dp7h_openshift-console-operator(a6a41630-31a9-4f8d-a321-be18df3b14e9)\"" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" podUID="a6a41630-31a9-4f8d-a321-be18df3b14e9"
Jul 23 11:27:26.535148 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:26.535117    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v2v4p"
Jul 23 11:27:28.988070 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:28.988021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:28.990617 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:28.990592    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1adbad02-1655-44ea-b057-5629f0df8c47-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q4hbv\" (UID: \"1adbad02-1655-44ea-b057-5629f0df8c47\") " pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:29.070051 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:29.070015    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfh54\""
Jul 23 11:27:29.078124 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:29.078095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q4hbv"
Jul 23 11:27:29.209638 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:29.209599    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q4hbv"]
Jul 23 11:27:29.212521 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:29.212499    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adbad02_1655_44ea_b057_5629f0df8c47.slice/crio-26ac3a9633b733e21f79bba193a4b52581d92630eecf4028bb233c83697c5a0d WatchSource:0}: Error finding container 26ac3a9633b733e21f79bba193a4b52581d92630eecf4028bb233c83697c5a0d: Status 404 returned error can't find the container with id 26ac3a9633b733e21f79bba193a4b52581d92630eecf4028bb233c83697c5a0d
Jul 23 11:27:29.691683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:29.691647    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4hbv" event={"ID":"1adbad02-1655-44ea-b057-5629f0df8c47","Type":"ContainerStarted","Data":"26ac3a9633b733e21f79bba193a4b52581d92630eecf4028bb233c83697c5a0d"}
Jul 23 11:27:31.913704 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.913660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:27:31.914116 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.913744    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:27:31.916757 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.916733    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 11:27:31.916899 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.916742    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 11:27:31.926536 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.926505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/14e709f1-29c4-4514-ac9f-35b04391b96a-original-pull-secret\") pod \"global-pull-secret-syncer-wzbq5\" (UID: \"14e709f1-29c4-4514-ac9f-35b04391b96a\") " pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:27:31.926536 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:31.926530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebe0b363-e249-4eb8-a373-8ab4ad5a5f11-metrics-certs\") pod \"network-metrics-daemon-4nbxv\" (UID: \"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11\") " pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:27:32.004016 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.003978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wzbq5"
Jul 23 11:27:32.013344 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.013319    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zh6tm\""
Jul 23 11:27:32.014675 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.014646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:32.017689 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.017639    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9jln\" (UniqueName: \"kubernetes.io/projected/fe8651cb-6ed7-433e-b2be-d7ae522dd03e-kube-api-access-f9jln\") pod \"network-check-target-mgkmf\" (UID: \"fe8651cb-6ed7-433e-b2be-d7ae522dd03e\") " pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:32.021194 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.021174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4nbxv"
Jul 23 11:27:32.318719 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.318641    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8j6tk\""
Jul 23 11:27:32.326738 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:32.326705    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:33.528919 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.528881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:33.529401 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.528991    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:33.531771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.531744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5992ce67-f59b-4fbd-ab61-0708b33410df-metrics-tls\") pod \"dns-default-vjptn\" (UID: \"5992ce67-f59b-4fbd-ab61-0708b33410df\") " pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:33.531900 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.531806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"image-registry-56d49f9bf-ftdc7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") " pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:33.728535 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.728506    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t2g4d\""
Jul 23 11:27:33.735964 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.735930    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:33.752371 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.752339    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d6qnx\""
Jul 23 11:27:33.759848 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.759812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:33.931956 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.931921    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:33.934476 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:33.934438    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d4e3a23-07a2-4660-9ec8-a1a43324d0f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-dgxv2\" (UID: \"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:34.032986 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.032945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:34.033155 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.033003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:34.033232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.033146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:34.033876 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.033853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-service-ca-bundle\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:34.035656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.035636    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702a7933-1e2c-4e40-9bc5-f0ed5ee0e139-metrics-certs\") pod \"router-default-54c8b47ff6-g6ndr\" (UID: \"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139\") " pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:34.035771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.035662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xnm87\" (UID: \"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:34.094998 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.094960    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jmphl\""
Jul 23 11:27:34.102374 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.102342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"
Jul 23 11:27:34.194232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.193956    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9sqq6\""
Jul 23 11:27:34.201506 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.201479    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"
Jul 23 11:27:34.237783 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.237743    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-62s7d\""
Jul 23 11:27:34.246069 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.246036    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:34.292971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.292945    2574 scope.go:117] "RemoveContainer" containerID="98188f0dbff03aca338bb9c0a1d0ea306dae9f99399e8856822300adbd6fcdd4"
Jul 23 11:27:34.824328 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.818746    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wzbq5"]
Jul 23 11:27:34.838436 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:34.831576    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e709f1_29c4_4514_ac9f_35b04391b96a.slice/crio-66ae55f5e6558628b95bcd5aa0cfd6beaedc44652457f93f7f92a596cb782a95 WatchSource:0}: Error finding container 66ae55f5e6558628b95bcd5aa0cfd6beaedc44652457f93f7f92a596cb782a95: Status 404 returned error can't find the container with id 66ae55f5e6558628b95bcd5aa0cfd6beaedc44652457f93f7f92a596cb782a95
Jul 23 11:27:34.907320 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.907256    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mgkmf"]
Jul 23 11:27:34.923759 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:34.923548    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8651cb_6ed7_433e_b2be_d7ae522dd03e.slice/crio-d006286c715f51c01f5556482d9b221c2e4e8367d68f756a3ded28912cd9983f WatchSource:0}: Error finding container d006286c715f51c01f5556482d9b221c2e4e8367d68f756a3ded28912cd9983f: Status 404 returned error can't find the container with id d006286c715f51c01f5556482d9b221c2e4e8367d68f756a3ded28912cd9983f
Jul 23 11:27:34.965908 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.965878    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87"]
Jul 23 11:27:34.971044 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:34.970998    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7112e7_a9f5_4bb3_b7cf_6fe9e6c0e47a.slice/crio-d03ab8e3c174009fee616c33e79336ef84df525e6aa4d06e80f92120cd8eeff9 WatchSource:0}: Error finding container d03ab8e3c174009fee616c33e79336ef84df525e6aa4d06e80f92120cd8eeff9: Status 404 returned error can't find the container with id d03ab8e3c174009fee616c33e79336ef84df525e6aa4d06e80f92120cd8eeff9
Jul 23 11:27:34.971606 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.971399    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:27:34.976170 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:34.976144    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c36f467_882d_4da8_87f8_c9414a008ad7.slice/crio-8ef1c548bc2a0bf825d36851d60b07dd617dbd2eb33628a49ff6189ef771733b WatchSource:0}: Error finding container 8ef1c548bc2a0bf825d36851d60b07dd617dbd2eb33628a49ff6189ef771733b: Status 404 returned error can't find the container with id 8ef1c548bc2a0bf825d36851d60b07dd617dbd2eb33628a49ff6189ef771733b
Jul 23 11:27:34.981997 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:34.981152    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vjptn"]
Jul 23 11:27:34.991792 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:34.991655    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5992ce67_f59b_4fbd_ab61_0708b33410df.slice/crio-ee5de2752a5b2b092d793fd2e1d26115876285f306c44e25f5edda117c1858a7 WatchSource:0}: Error finding container ee5de2752a5b2b092d793fd2e1d26115876285f306c44e25f5edda117c1858a7: Status 404 returned error can't find the container with id ee5de2752a5b2b092d793fd2e1d26115876285f306c44e25f5edda117c1858a7
Jul 23 11:27:35.018829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.018798    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4nbxv"]
Jul 23 11:27:35.021872 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:35.021849    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe0b363_e249_4eb8_a373_8ab4ad5a5f11.slice/crio-a0f291e7171b551427143a58a5ac35784dd3d31cc198843a429600116d9c1d38 WatchSource:0}: Error finding container a0f291e7171b551427143a58a5ac35784dd3d31cc198843a429600116d9c1d38: Status 404 returned error can't find the container with id a0f291e7171b551427143a58a5ac35784dd3d31cc198843a429600116d9c1d38
Jul 23 11:27:35.225675 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.225578    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54c8b47ff6-g6ndr"]
Jul 23 11:27:35.227105 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.227025    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2"]
Jul 23 11:27:35.228890 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:35.228864    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702a7933_1e2c_4e40_9bc5_f0ed5ee0e139.slice/crio-be06eef04190982c97546caa76fdc3072491231aae919ac25557d0b21dea9988 WatchSource:0}: Error finding container be06eef04190982c97546caa76fdc3072491231aae919ac25557d0b21dea9988: Status 404 returned error can't find the container with id be06eef04190982c97546caa76fdc3072491231aae919ac25557d0b21dea9988
Jul 23 11:27:35.229731 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:35.229709    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4e3a23_07a2_4660_9ec8_a1a43324d0f0.slice/crio-3fe8d810687de155bba0601b6bce73dc49be1eac5fcdb61fcf651e98c7d1fb1e WatchSource:0}: Error finding container 3fe8d810687de155bba0601b6bce73dc49be1eac5fcdb61fcf651e98c7d1fb1e: Status 404 returned error can't find the container with id 3fe8d810687de155bba0601b6bce73dc49be1eac5fcdb61fcf651e98c7d1fb1e
Jul 23 11:27:35.736716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.736039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" event={"ID":"0c36f467-882d-4da8-87f8-c9414a008ad7","Type":"ContainerStarted","Data":"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"}
Jul 23 11:27:35.736716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.736099    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" event={"ID":"0c36f467-882d-4da8-87f8-c9414a008ad7","Type":"ContainerStarted","Data":"8ef1c548bc2a0bf825d36851d60b07dd617dbd2eb33628a49ff6189ef771733b"}
Jul 23 11:27:35.736716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.736451    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:35.738883 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.738789    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4hbv" event={"ID":"1adbad02-1655-44ea-b057-5629f0df8c47","Type":"ContainerStarted","Data":"5103ea645bdd3d0803dc2128db41eb297b00c98e99b2e6c5f04df2f60de9bc03"}
Jul 23 11:27:35.743628 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.743558    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4nbxv" event={"ID":"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11","Type":"ContainerStarted","Data":"a0f291e7171b551427143a58a5ac35784dd3d31cc198843a429600116d9c1d38"}
Jul 23 11:27:35.745105 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.745041    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2" event={"ID":"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0","Type":"ContainerStarted","Data":"3fe8d810687de155bba0601b6bce73dc49be1eac5fcdb61fcf651e98c7d1fb1e"}
Jul 23 11:27:35.746703 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.746643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vjptn" event={"ID":"5992ce67-f59b-4fbd-ab61-0708b33410df","Type":"ContainerStarted","Data":"ee5de2752a5b2b092d793fd2e1d26115876285f306c44e25f5edda117c1858a7"}
Jul 23 11:27:35.752445 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.750247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:27:35.752445 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.750322    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" event={"ID":"a6a41630-31a9-4f8d-a321-be18df3b14e9","Type":"ContainerStarted","Data":"9e9eb641e061afcf95fb5357dd1ddb9d5d333579d147eab90acf99a407eed521"}
Jul 23 11:27:35.752445 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.751169    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:35.753708 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.753642    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87" event={"ID":"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a","Type":"ContainerStarted","Data":"d03ab8e3c174009fee616c33e79336ef84df525e6aa4d06e80f92120cd8eeff9"}
Jul 23 11:27:35.758573 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.756693    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" podStartSLOduration=68.756677716 podStartE2EDuration="1m8.756677716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:35.754999591 +0000 UTC m=+70.040465353" watchObservedRunningTime="2026-07-23 11:27:35.756677716 +0000 UTC m=+70.042143481"
Jul 23 11:27:35.760540 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.759912    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mgkmf" event={"ID":"fe8651cb-6ed7-433e-b2be-d7ae522dd03e","Type":"ContainerStarted","Data":"fe0cb5e8c33ae8a8e6e3b72e56fe0a99d7f8e3987d4100c0f1477120bd978e31"}
Jul 23 11:27:35.760540 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.759954    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mgkmf" event={"ID":"fe8651cb-6ed7-433e-b2be-d7ae522dd03e","Type":"ContainerStarted","Data":"d006286c715f51c01f5556482d9b221c2e4e8367d68f756a3ded28912cd9983f"}
Jul 23 11:27:35.760540 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.760478    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:27:35.764882 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.764824    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wzbq5" event={"ID":"14e709f1-29c4-4514-ac9f-35b04391b96a","Type":"ContainerStarted","Data":"66ae55f5e6558628b95bcd5aa0cfd6beaedc44652457f93f7f92a596cb782a95"}
Jul 23 11:27:35.772207 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.772159    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h"
Jul 23 11:27:35.773925 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.773869    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-4dp7h" podStartSLOduration=25.855787475 podStartE2EDuration="33.77384593s" totalImagesPullingTime="7.918058455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:35.772356508 +0000 UTC m=+70.057822272" watchObservedRunningTime="2026-07-23 11:27:35.77384593 +0000 UTC m=+70.059311697"
Jul 23 11:27:35.778280 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.777696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr" event={"ID":"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139","Type":"ContainerStarted","Data":"7a827690ba1668d50ff48ffe08a6ff600d96cc9ac2620b13b2edd4c86ecfea33"}
Jul 23 11:27:35.778280 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.777741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr" event={"ID":"702a7933-1e2c-4e40-9bc5-f0ed5ee0e139","Type":"ContainerStarted","Data":"be06eef04190982c97546caa76fdc3072491231aae919ac25557d0b21dea9988"}
Jul 23 11:27:35.819953 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.819345    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mgkmf" podStartSLOduration=69.81932765 podStartE2EDuration="1m9.81932765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:35.817961462 +0000 UTC m=+70.103427228" watchObservedRunningTime="2026-07-23 11:27:35.81932765 +0000 UTC m=+70.104793399"
Jul 23 11:27:35.839501 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:35.838818    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr" podStartSLOduration=33.838786827 podStartE2EDuration="33.838786827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:35.83833822 +0000 UTC m=+70.123803984" watchObservedRunningTime="2026-07-23 11:27:35.838786827 +0000 UTC m=+70.124252578"
Jul 23 11:27:36.243068 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.243036    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"]
Jul 23 11:27:36.248138 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.248106    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:36.248288 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.248264    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"]
Jul 23 11:27:36.248373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.248215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.252024 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.251990    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:36.252024 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.252026    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"]
Jul 23 11:27:36.255295 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.255269    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 23 11:27:36.257092 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.257053    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 23 11:27:36.258870 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.258815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.267894 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.267862    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 23 11:27:36.274588 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.274343    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 23 11:27:36.276310 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.275342    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 23 11:27:36.280489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.276679    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 23 11:27:36.280489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.278222    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 23 11:27:36.282371 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.282354    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 23 11:27:36.286964 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.286946    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"]
Jul 23 11:27:36.287683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.287106    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"]
Jul 23 11:27:36.287683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.287132    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"]
Jul 23 11:27:36.287683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.287217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.294782 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.294765    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 23 11:27:36.295823 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.294785    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l5zk6\""
Jul 23 11:27:36.306287 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.306263    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-fz5kc"]
Jul 23 11:27:36.314759 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.314738    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:27:36.320522 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.320398    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 11:27:36.321569 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.321551    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-b2qjq\""
Jul 23 11:27:36.322276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.322188    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 11:27:36.326374 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.326353    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fz5kc"]
Jul 23 11:27:36.370538 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/298e8453-264b-4183-b8d7-3e180a50e972-tmp\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.370538 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370538    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.370767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.370767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.370767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc662\" (UniqueName: \"kubernetes.io/projected/017318e0-8a56-495f-8a56-bb513af9fc6c-kube-api-access-hc662\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.370767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.370767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/017318e0-8a56-495f-8a56-bb513af9fc6c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.371009 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/298e8453-264b-4183-b8d7-3e180a50e972-klusterlet-config\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.371009 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.370828    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49z4z\" (UniqueName: \"kubernetes.io/projected/298e8453-264b-4183-b8d7-3e180a50e972-kube-api-access-49z4z\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.473114 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473051    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/298e8453-264b-4183-b8d7-3e180a50e972-tmp\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.473574 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.472045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/298e8453-264b-4183-b8d7-3e180a50e972-tmp\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.473705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473629    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aab5a2c9-68e4-4362-86ac-c46c77935801-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.473705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.473817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.473817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.473817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc662\" (UniqueName: \"kubernetes.io/projected/017318e0-8a56-495f-8a56-bb513af9fc6c-kube-api-access-hc662\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.474010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.474010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/017318e0-8a56-495f-8a56-bb513af9fc6c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.474010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/298e8453-264b-4183-b8d7-3e180a50e972-klusterlet-config\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.474010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49z4z\" (UniqueName: \"kubernetes.io/projected/298e8453-264b-4183-b8d7-3e180a50e972-kube-api-access-49z4z\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.474010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.473944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bnbg\" (UniqueName: \"kubernetes.io/projected/aab5a2c9-68e4-4362-86ac-c46c77935801-kube-api-access-7bnbg\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.476683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.476662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/017318e0-8a56-495f-8a56-bb513af9fc6c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.476942 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.476914    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7kds\" (UniqueName: \"kubernetes.io/projected/41641bf8-7229-46c8-9881-6b111a01ff21-kube-api-access-h7kds\") pod \"downloads-5bc67dcd46-fz5kc\" (UID: \"41641bf8-7229-46c8-9881-6b111a01ff21\") " pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:27:36.479668 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.479494    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.480062 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.480015    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.482840 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.482799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/298e8453-264b-4183-b8d7-3e180a50e972-klusterlet-config\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.486517 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.486457    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-ca\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.490264 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.490239    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/017318e0-8a56-495f-8a56-bb513af9fc6c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.500433 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.500340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc662\" (UniqueName: \"kubernetes.io/projected/017318e0-8a56-495f-8a56-bb513af9fc6c-kube-api-access-hc662\") pod \"cluster-proxy-proxy-agent-78f55b6695-pjmbh\" (UID: \"017318e0-8a56-495f-8a56-bb513af9fc6c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.502855 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.502583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49z4z\" (UniqueName: \"kubernetes.io/projected/298e8453-264b-4183-b8d7-3e180a50e972-kube-api-access-49z4z\") pod \"klusterlet-addon-workmgr-68c5d5957f-lvplr\" (UID: \"298e8453-264b-4183-b8d7-3e180a50e972\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.578617 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.577730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bnbg\" (UniqueName: \"kubernetes.io/projected/aab5a2c9-68e4-4362-86ac-c46c77935801-kube-api-access-7bnbg\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.578617 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.577822    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7kds\" (UniqueName: \"kubernetes.io/projected/41641bf8-7229-46c8-9881-6b111a01ff21-kube-api-access-h7kds\") pod \"downloads-5bc67dcd46-fz5kc\" (UID: \"41641bf8-7229-46c8-9881-6b111a01ff21\") " pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:27:36.578617 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.577881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aab5a2c9-68e4-4362-86ac-c46c77935801-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.583012 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.582921    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/aab5a2c9-68e4-4362-86ac-c46c77935801-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.595303 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.595031    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"
Jul 23 11:27:36.607337 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.607157    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:36.610479 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.610388    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bnbg\" (UniqueName: \"kubernetes.io/projected/aab5a2c9-68e4-4362-86ac-c46c77935801-kube-api-access-7bnbg\") pod \"managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29\" (UID: \"aab5a2c9-68e4-4362-86ac-c46c77935801\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.611609 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.611507    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7kds\" (UniqueName: \"kubernetes.io/projected/41641bf8-7229-46c8-9881-6b111a01ff21-kube-api-access-h7kds\") pod \"downloads-5bc67dcd46-fz5kc\" (UID: \"41641bf8-7229-46c8-9881-6b111a01ff21\") " pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:27:36.620338 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.620314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"
Jul 23 11:27:36.645565 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.645260    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:27:36.785495 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.785355    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:36.787357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:36.787151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54c8b47ff6-g6ndr"
Jul 23 11:27:40.950662 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:40.950621    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"]
Jul 23 11:27:42.462544 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:42.462504    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298e8453_264b_4183_b8d7_3e180a50e972.slice/crio-d8fff5bb4f69bc23a4ace29407a8ce9ec14fc19d952d7c11a097fd76656520e2 WatchSource:0}: Error finding container d8fff5bb4f69bc23a4ace29407a8ce9ec14fc19d952d7c11a097fd76656520e2: Status 404 returned error can't find the container with id d8fff5bb4f69bc23a4ace29407a8ce9ec14fc19d952d7c11a097fd76656520e2
Jul 23 11:27:42.808669 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:42.808574    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr" event={"ID":"298e8453-264b-4183-b8d7-3e180a50e972","Type":"ContainerStarted","Data":"d8fff5bb4f69bc23a4ace29407a8ce9ec14fc19d952d7c11a097fd76656520e2"}
Jul 23 11:27:43.695755 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.695719    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-fz5kc"]
Jul 23 11:27:43.710415 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.710377    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29"]
Jul 23 11:27:43.734146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.734113    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh"]
Jul 23 11:27:43.785836 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:43.785801    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41641bf8_7229_46c8_9881_6b111a01ff21.slice/crio-1d116f2477a46296d82b8483f430cb1f95eede25375166a9fde4a14e95c6a8f0 WatchSource:0}: Error finding container 1d116f2477a46296d82b8483f430cb1f95eede25375166a9fde4a14e95c6a8f0: Status 404 returned error can't find the container with id 1d116f2477a46296d82b8483f430cb1f95eede25375166a9fde4a14e95c6a8f0
Jul 23 11:27:43.790105 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:27:43.790076    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod017318e0_8a56_495f_8a56_bb513af9fc6c.slice/crio-18d994a80158ccd243f9d6e3bbcf50499c27ab4040c36b92d8b86ddc7747656c WatchSource:0}: Error finding container 18d994a80158ccd243f9d6e3bbcf50499c27ab4040c36b92d8b86ddc7747656c: Status 404 returned error can't find the container with id 18d994a80158ccd243f9d6e3bbcf50499c27ab4040c36b92d8b86ddc7747656c
Jul 23 11:27:43.815553 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.815453    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh" event={"ID":"017318e0-8a56-495f-8a56-bb513af9fc6c","Type":"ContainerStarted","Data":"18d994a80158ccd243f9d6e3bbcf50499c27ab4040c36b92d8b86ddc7747656c"}
Jul 23 11:27:43.818026 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.817942    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4nbxv" event={"ID":"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11","Type":"ContainerStarted","Data":"59294c8d5158cdbaa10f80332c3a1e22655cc36ffb1b2c1c9c1ac0019d99d2f2"}
Jul 23 11:27:43.819890 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.819832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fz5kc" event={"ID":"41641bf8-7229-46c8-9881-6b111a01ff21","Type":"ContainerStarted","Data":"1d116f2477a46296d82b8483f430cb1f95eede25375166a9fde4a14e95c6a8f0"}
Jul 23 11:27:43.821297 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.821231    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29" event={"ID":"aab5a2c9-68e4-4362-86ac-c46c77935801","Type":"ContainerStarted","Data":"84579b373e2d6c5b8f615b3ce43291d27cd62deb5beef4e97dc408c73e5f0cad"}
Jul 23 11:27:43.822975 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.822948    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87" event={"ID":"0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a","Type":"ContainerStarted","Data":"efcfcc9250ce35ae804a7ec6da8a1de5101961b24b9b10a0a21b1ad057f50933"}
Jul 23 11:27:43.839695 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:43.839503    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xnm87" podStartSLOduration=33.618422267 podStartE2EDuration="41.83948091s" totalImagesPullingTime="8.221058643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:43.838080967 +0000 UTC m=+78.123546733" watchObservedRunningTime="2026-07-23 11:27:43.83948091 +0000 UTC m=+78.124946675"
Jul 23 11:27:44.848107 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.848058    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4hbv" event={"ID":"1adbad02-1655-44ea-b057-5629f0df8c47","Type":"ContainerStarted","Data":"ddfa98789090e9413a191224f6a1138a0374eac732b4e40dadd9598aa1b0caa1"}
Jul 23 11:27:44.855102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.854588    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4nbxv" event={"ID":"ebe0b363-e249-4eb8-a373-8ab4ad5a5f11","Type":"ContainerStarted","Data":"975c51c473111723a99fa9f4afb2cc0eccf28d51886953b7db2e4652cd6e93d1"}
Jul 23 11:27:44.860596 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.859338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2" event={"ID":"9d4e3a23-07a2-4660-9ec8-a1a43324d0f0","Type":"ContainerStarted","Data":"aa52a1044c43d983691e892aaa3556949c6796bc2e6a08d2b959b2cee0ede20a"}
Jul 23 11:27:44.869504 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.868897    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vjptn" event={"ID":"5992ce67-f59b-4fbd-ab61-0708b33410df","Type":"ContainerStarted","Data":"a1f270df884fb9cdf2997c05a678e8d08c1da75d1485af19ab9c4e62e24f5c26"}
Jul 23 11:27:44.869504 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.868950    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vjptn" event={"ID":"5992ce67-f59b-4fbd-ab61-0708b33410df","Type":"ContainerStarted","Data":"68f9889ff370dcc9f265276abefc59abf7133bd7c7fd2357b51e00001ae4ce78"}
Jul 23 11:27:44.869893 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.869833    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:44.872899 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.872845    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wzbq5" event={"ID":"14e709f1-29c4-4514-ac9f-35b04391b96a","Type":"ContainerStarted","Data":"d61f1a5cdd45d36132ec0618bb18bc03b0cbbd15c532640392099e1a308a8613"}
Jul 23 11:27:44.880394 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.878660    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4nbxv" podStartSLOduration=71.441079261 podStartE2EDuration="1m18.878642485s" totalImagesPullingTime="7.437563224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:44.878108219 +0000 UTC m=+79.163573987" watchObservedRunningTime="2026-07-23 11:27:44.878642485 +0000 UTC m=+79.164108252"
Jul 23 11:27:44.910245 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.907593    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vjptn" podStartSLOduration=35.709604197 podStartE2EDuration="43.907568782s" totalImagesPullingTime="8.197964585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:44.906550379 +0000 UTC m=+79.192016146" watchObservedRunningTime="2026-07-23 11:27:44.907568782 +0000 UTC m=+79.193034546"
Jul 23 11:27:44.942005 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:44.941939    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-dgxv2" podStartSLOduration=35.656169465 podStartE2EDuration="43.941917571s" totalImagesPullingTime="8.285748106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:44.9412435 +0000 UTC m=+79.226709267" watchObservedRunningTime="2026-07-23 11:27:44.941917571 +0000 UTC m=+79.227383338"
Jul 23 11:27:53.741551 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.741506    2574 patch_prober.go:28] interesting pod/image-registry-56d49f9bf-ftdc7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 11:27:53.742039 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.741579    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 11:27:53.931101 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.931059    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29" event={"ID":"aab5a2c9-68e4-4362-86ac-c46c77935801","Type":"ContainerStarted","Data":"ba5b3dbf02b4aff47c64718649a57b844002134c67ce67dd93e2e7dfac4ca1ee"}
Jul 23 11:27:53.933756 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.933728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh" event={"ID":"017318e0-8a56-495f-8a56-bb513af9fc6c","Type":"ContainerStarted","Data":"774599769a341f1092a68579906a61eb5bb9a796cec28c3cc3b8a085c27decc9"}
Jul 23 11:27:53.933890 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.933773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh" event={"ID":"017318e0-8a56-495f-8a56-bb513af9fc6c","Type":"ContainerStarted","Data":"aa559d4d199651d961a73c96e9aed12d3236d1be27f9f61944002e2c6ae25708"}
Jul 23 11:27:53.933890 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.933791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh" event={"ID":"017318e0-8a56-495f-8a56-bb513af9fc6c","Type":"ContainerStarted","Data":"7a311aef5fb65cf7858cbd81119d3c143e41af0a61bc7bde50befb6962c2353d"}
Jul 23 11:27:53.936188 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.936150    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q4hbv" event={"ID":"1adbad02-1655-44ea-b057-5629f0df8c47","Type":"ContainerStarted","Data":"3f5a146646b995b9239d9a975f90558190040761f0c791665f419dad31e504b6"}
Jul 23 11:27:53.937699 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.937666    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr" event={"ID":"298e8453-264b-4183-b8d7-3e180a50e972","Type":"ContainerStarted","Data":"697b6a8cbfa7929d3508a394bfefa016d6d700dcf13384cb23b0e0d4250b8eee"}
Jul 23 11:27:53.937894 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.937871    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:53.939983 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.939964    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr"
Jul 23 11:27:53.946810 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.946764    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7bfd7bbc4d-tbb29" podStartSLOduration=8.723730432 podStartE2EDuration="17.946746325s" totalImagesPullingTime="9.223015893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:53.945296741 +0000 UTC m=+88.230762506" watchObservedRunningTime="2026-07-23 11:27:53.946746325 +0000 UTC m=+88.232212093"
Jul 23 11:27:53.946954 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.946846    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wzbq5" podStartSLOduration=79.240284796 podStartE2EDuration="1m27.946841853s" totalImagesPullingTime="8.706557057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:44.97190183 +0000 UTC m=+79.257367598" watchObservedRunningTime="2026-07-23 11:27:53.946841853 +0000 UTC m=+88.232307653"
Jul 23 11:27:53.965344 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.965289    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q4hbv" podStartSLOduration=17.476552182 podStartE2EDuration="40.965267786s" totalImagesPullingTime="23.488715604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 11:27:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:53.963840869 +0000 UTC m=+88.249306636" watchObservedRunningTime="2026-07-23 11:27:53.965267786 +0000 UTC m=+88.250733551"
Jul 23 11:27:53.981486 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.981413    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78f55b6695-pjmbh" podStartSLOduration=8.736707147 podStartE2EDuration="17.981394669s" totalImagesPullingTime="9.244687522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:53.98051554 +0000 UTC m=+88.265981307" watchObservedRunningTime="2026-07-23 11:27:53.981394669 +0000 UTC m=+88.266860436"
Jul 23 11:27:53.996165 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:53.996069    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68c5d5957f-lvplr" podStartSLOduration=7.397755812 podStartE2EDuration="17.996051871s" totalImagesPullingTime="10.598296059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:27:53.995604967 +0000 UTC m=+88.281070734" watchObservedRunningTime="2026-07-23 11:27:53.996051871 +0000 UTC m=+88.281517636"
Jul 23 11:27:55.898534 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:55.898499    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vjptn"
Jul 23 11:27:56.790420 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:56.789768    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:27:58.923078 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:58.923037    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-27dxq"]
Jul 23 11:27:59.028334 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.028304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.032524 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.032495    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 11:27:59.032694 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.032635    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 11:27:59.033554 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.033532    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m86hz\""
Jul 23 11:27:59.033714 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.033672    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 11:27:59.033833 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.033749    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 11:27:59.085241 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085204    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-tls\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-wtmp\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085296    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-root\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-textfile\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/2c81da6d-2852-4357-b28c-adfa804e7ad3-kube-api-access-l77bk\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085375    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-accelerators-collector-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085766 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085420    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-sys\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.085766 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.085436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-metrics-client-ca\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-wtmp\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186306    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-root\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-textfile\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186384    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/2c81da6d-2852-4357-b28c-adfa804e7ad3-kube-api-access-l77bk\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186415    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-accelerators-collector-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-sys\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-metrics-client-ca\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186541    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-tls\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.186933 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.186801    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-sys\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.187265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.187150    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-root\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.187265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.187259    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-wtmp\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.187265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.187325    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-accelerators-collector-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.187602 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.187400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-textfile\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.187837 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.187812    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c81da6d-2852-4357-b28c-adfa804e7ad3-metrics-client-ca\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.189296 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.189274    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.189395 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.189309    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2c81da6d-2852-4357-b28c-adfa804e7ad3-node-exporter-tls\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.198498 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.198450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77bk\" (UniqueName: \"kubernetes.io/projected/2c81da6d-2852-4357-b28c-adfa804e7ad3-kube-api-access-l77bk\") pod \"node-exporter-27dxq\" (UID: \"2c81da6d-2852-4357-b28c-adfa804e7ad3\") " pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:27:59.341275 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:27:59.341239    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-27dxq"
Jul 23 11:28:01.004153 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:01.004111    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:28:03.713516 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:28:03.713454    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c81da6d_2852_4357_b28c_adfa804e7ad3.slice/crio-3373952a325e6499855e3aabb6a782ce8356e1c7a459212c00db57a0f621c5e9 WatchSource:0}: Error finding container 3373952a325e6499855e3aabb6a782ce8356e1c7a459212c00db57a0f621c5e9: Status 404 returned error can't find the container with id 3373952a325e6499855e3aabb6a782ce8356e1c7a459212c00db57a0f621c5e9
Jul 23 11:28:03.988767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:03.988741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27dxq" event={"ID":"2c81da6d-2852-4357-b28c-adfa804e7ad3","Type":"ContainerStarted","Data":"3373952a325e6499855e3aabb6a782ce8356e1c7a459212c00db57a0f621c5e9"}
Jul 23 11:28:04.996059 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:04.996012    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-fz5kc" event={"ID":"41641bf8-7229-46c8-9881-6b111a01ff21","Type":"ContainerStarted","Data":"8de2165ec3e33c7ceb42d02499a5140193d517dba51445231a0276a521d33432"}
Jul 23 11:28:04.996566 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:04.996341    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:28:05.013373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:05.013342    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-fz5kc"
Jul 23 11:28:05.014389 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:05.014346    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-fz5kc" podStartSLOduration=8.866507777 podStartE2EDuration="29.014330952s" totalImagesPullingTime="20.147823175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:28:05.013017057 +0000 UTC m=+99.298482845" watchObservedRunningTime="2026-07-23 11:28:05.014330952 +0000 UTC m=+99.299796718"
Jul 23 11:28:06.003712 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:06.003617    2574 generic.go:356] "Generic (PLEG): container finished" podID="2c81da6d-2852-4357-b28c-adfa804e7ad3" containerID="40aea294a96c5e3a3c9c1e6ff3d2bb62ab63d30d468df76f2124215343445e62" exitCode=0
Jul 23 11:28:06.004176 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:06.003710    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27dxq" event={"ID":"2c81da6d-2852-4357-b28c-adfa804e7ad3","Type":"ContainerDied","Data":"40aea294a96c5e3a3c9c1e6ff3d2bb62ab63d30d468df76f2124215343445e62"}
Jul 23 11:28:07.013737 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:07.013702    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27dxq" event={"ID":"2c81da6d-2852-4357-b28c-adfa804e7ad3","Type":"ContainerStarted","Data":"c0a6e89d760d04d9cdd80930916b8cff7da0d80c056f8bcdd1177de613cabd83"}
Jul 23 11:28:07.014229 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:07.013807    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-27dxq" event={"ID":"2c81da6d-2852-4357-b28c-adfa804e7ad3","Type":"ContainerStarted","Data":"92008761767f91a6477e08f766befecbfd89e4f890776424045f8e0493043c28"}
Jul 23 11:28:07.048725 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:07.048669    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-27dxq" podStartSLOduration=7.268473221 podStartE2EDuration="9.048651759s" totalImagesPullingTime="1.664805691s" totalInitContainerRuntime="115.372847ms" isStatefulPod=true podCreationTimestamp="2026-07-23 11:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:28:07.046796847 +0000 UTC m=+101.332262613" watchObservedRunningTime="2026-07-23 11:28:07.048651759 +0000 UTC m=+101.334117525"
Jul 23 11:28:07.791524 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:07.791487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mgkmf"
Jul 23 11:28:26.027296 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.027237    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerName="registry" containerID="cri-o://e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9" gracePeriod=30
Jul 23 11:28:26.316484 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.316444    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:28:26.336053 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336024    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336062    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336086    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336408 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336240    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336408 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336311    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336408 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336360    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336408 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdx9x\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x\") pod \"0c36f467-882d-4da8-87f8-c9414a008ad7\" (UID: \"0c36f467-882d-4da8-87f8-c9414a008ad7\") "
Jul 23 11:28:26.336709 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.336683    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:28:26.338718 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.338669    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:28:26.339011 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.338982    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:28:26.339146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.339064    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x" (OuterVolumeSpecName: "kube-api-access-cdx9x") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "kube-api-access-cdx9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:28:26.339146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.339117    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:28:26.339231 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.339146    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:28:26.339420 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.339390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:28:26.346691 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.346661    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0c36f467-882d-4da8-87f8-c9414a008ad7" (UID: "0c36f467-882d-4da8-87f8-c9414a008ad7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:28:26.437426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437384    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c36f467-882d-4da8-87f8-c9414a008ad7-ca-trust-extracted\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437420    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-trusted-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437429    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdx9x\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-kube-api-access-cdx9x\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437438    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-bound-sa-token\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437449    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-image-registry-private-configuration\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437458    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-certificates\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437496    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c36f467-882d-4da8-87f8-c9414a008ad7-installation-pull-secrets\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:26.437707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:26.437509    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c36f467-882d-4da8-87f8-c9414a008ad7-registry-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:28:27.099572 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.099534    2574 generic.go:356] "Generic (PLEG): container finished" podID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerID="e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9" exitCode=0
Jul 23 11:28:27.099971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.099580    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7"
Jul 23 11:28:27.099971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.099614    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" event={"ID":"0c36f467-882d-4da8-87f8-c9414a008ad7","Type":"ContainerDied","Data":"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"}
Jul 23 11:28:27.099971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.099663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d49f9bf-ftdc7" event={"ID":"0c36f467-882d-4da8-87f8-c9414a008ad7","Type":"ContainerDied","Data":"8ef1c548bc2a0bf825d36851d60b07dd617dbd2eb33628a49ff6189ef771733b"}
Jul 23 11:28:27.099971 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.099685    2574 scope.go:117] "RemoveContainer" containerID="e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"
Jul 23 11:28:27.108698 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.108681    2574 scope.go:117] "RemoveContainer" containerID="e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"
Jul 23 11:28:27.109004 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:28:27.108984    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9\": container with ID starting with e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9 not found: ID does not exist" containerID="e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"
Jul 23 11:28:27.109057 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.109013    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9"} err="failed to get container status \"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9\": rpc error: code = NotFound desc = could not find container \"e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9\": container with ID starting with e5012c128ea483b0364ceab49bbf9720132ce718866d11d470a8e32a71b2d2a9 not found: ID does not exist"
Jul 23 11:28:27.120916 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.120891    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:28:27.126945 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:27.126922    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56d49f9bf-ftdc7"]
Jul 23 11:28:28.298008 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:28.297974    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" path="/var/lib/kubelet/pods/0c36f467-882d-4da8-87f8-c9414a008ad7/volumes"
Jul 23 11:28:33.125529 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:33.125495    2574 generic.go:356] "Generic (PLEG): container finished" podID="b0f4ac95-4fd6-4e47-8327-c7c6bf843339" containerID="8193b723a3af662718582014459414f6f78ced9281e9de6b8ba4a66708ce21e3" exitCode=0
Jul 23 11:28:33.125973 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:33.125547    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p" event={"ID":"b0f4ac95-4fd6-4e47-8327-c7c6bf843339","Type":"ContainerDied","Data":"8193b723a3af662718582014459414f6f78ced9281e9de6b8ba4a66708ce21e3"}
Jul 23 11:28:33.125973 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:33.125857    2574 scope.go:117] "RemoveContainer" containerID="8193b723a3af662718582014459414f6f78ced9281e9de6b8ba4a66708ce21e3"
Jul 23 11:28:34.131734 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:34.131696    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jdb9p" event={"ID":"b0f4ac95-4fd6-4e47-8327-c7c6bf843339","Type":"ContainerStarted","Data":"5aea843e51ed69fab1d7582e446981b04cc089059a8b6b7b801cd3fcc4919a24"}
Jul 23 11:28:38.148627 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.148595    2574 generic.go:356] "Generic (PLEG): container finished" podID="90b4c78f-dfd7-4365-8510-c0bf3a7acac8" containerID="9161a167f17fe0ac532242c2479dc8a5d5b63b5ae5905365d9f39119a889ead6" exitCode=0
Jul 23 11:28:38.149019 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.148665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-g295d" event={"ID":"90b4c78f-dfd7-4365-8510-c0bf3a7acac8","Type":"ContainerDied","Data":"9161a167f17fe0ac532242c2479dc8a5d5b63b5ae5905365d9f39119a889ead6"}
Jul 23 11:28:38.149069 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.149027    2574 scope.go:117] "RemoveContainer" containerID="9161a167f17fe0ac532242c2479dc8a5d5b63b5ae5905365d9f39119a889ead6"
Jul 23 11:28:38.150304 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.150284    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea610113-71c0-45ed-b083-edebe6288113" containerID="85f001eb6076ba78b2323b0768c84e992f138eaae5743df7a8d07ec6da79d73a" exitCode=0
Jul 23 11:28:38.150381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.150345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x" event={"ID":"ea610113-71c0-45ed-b083-edebe6288113","Type":"ContainerDied","Data":"85f001eb6076ba78b2323b0768c84e992f138eaae5743df7a8d07ec6da79d73a"}
Jul 23 11:28:38.150616 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:38.150604    2574 scope.go:117] "RemoveContainer" containerID="85f001eb6076ba78b2323b0768c84e992f138eaae5743df7a8d07ec6da79d73a"
Jul 23 11:28:39.159788 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:39.159748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-g295d" event={"ID":"90b4c78f-dfd7-4365-8510-c0bf3a7acac8","Type":"ContainerStarted","Data":"139f01fb3075520a57755d361efd95e28fc5e0dc0f7f68ee02e5cc8bda7d0cc9"}
Jul 23 11:28:39.161895 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:28:39.161862    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-zzq4x" event={"ID":"ea610113-71c0-45ed-b083-edebe6288113","Type":"ContainerStarted","Data":"845b1e00857a843c3042f50679c0e60970b35c3f9603e933ed16e10f2aeef253"}
Jul 23 11:30:50.787616 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.787530    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"]
Jul 23 11:30:50.788067 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.787837    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerName="registry"
Jul 23 11:30:50.788067 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.787847    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerName="registry"
Jul 23 11:30:50.788067 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.787895    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c36f467-882d-4da8-87f8-c9414a008ad7" containerName="registry"
Jul 23 11:30:50.790778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.790761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.796521 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.796494    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 11:30:50.796701 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.796682    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 11:30:50.796751 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.796684    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5hr4d\""
Jul 23 11:30:50.797899 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.797882    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 11:30:50.798006 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.797991    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 11:30:50.811236 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.811214    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-tmp\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.811342 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.811245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.811342 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.811264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b5n8\" (UniqueName: \"kubernetes.io/projected/90a30124-5518-4f73-8520-95aeb0d75a20-kube-api-access-9b5n8\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.811342 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.811288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-webhook-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.811513 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.811384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-manifests\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.860806 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.860772    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"]
Jul 23 11:30:50.911898 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.911863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-manifests\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.912134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.911946    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-tmp\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.912134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.911964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.912134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.911982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b5n8\" (UniqueName: \"kubernetes.io/projected/90a30124-5518-4f73-8520-95aeb0d75a20-kube-api-access-9b5n8\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.912134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.911999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-webhook-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.912365 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.912332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-manifests\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.914739 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.914706    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-webhook-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.914844 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.914754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90a30124-5518-4f73-8520-95aeb0d75a20-tmp\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.914844 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.914776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/90a30124-5518-4f73-8520-95aeb0d75a20-apiservice-cert\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:50.940175 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:50.940148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b5n8\" (UniqueName: \"kubernetes.io/projected/90a30124-5518-4f73-8520-95aeb0d75a20-kube-api-access-9b5n8\") pod \"opendatahub-operator-controller-manager-674bfcc66c-z5r7z\" (UID: \"90a30124-5518-4f73-8520-95aeb0d75a20\") " pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:51.100823 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:51.100786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:51.242699 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:51.242675    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"]
Jul 23 11:30:51.245442 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:30:51.245418    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a30124_5518_4f73_8520_95aeb0d75a20.slice/crio-b2da62388409c151a9775b34cc6c6e682138b7d7b7b4f8f1e480dd1b9ac63de0 WatchSource:0}: Error finding container b2da62388409c151a9775b34cc6c6e682138b7d7b7b4f8f1e480dd1b9ac63de0: Status 404 returned error can't find the container with id b2da62388409c151a9775b34cc6c6e682138b7d7b7b4f8f1e480dd1b9ac63de0
Jul 23 11:30:51.686604 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:51.686562    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z" event={"ID":"90a30124-5518-4f73-8520-95aeb0d75a20","Type":"ContainerStarted","Data":"b2da62388409c151a9775b34cc6c6e682138b7d7b7b4f8f1e480dd1b9ac63de0"}
Jul 23 11:30:54.703414 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:54.703375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z" event={"ID":"90a30124-5518-4f73-8520-95aeb0d75a20","Type":"ContainerStarted","Data":"ef0b80ddfc6f0733f997304d1fde7be45c2b914febda0c4b58aa853320029ff5"}
Jul 23 11:30:55.708654 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:55.708621    2574 generic.go:356] "Generic (PLEG): container finished" podID="90a30124-5518-4f73-8520-95aeb0d75a20" containerID="ef0b80ddfc6f0733f997304d1fde7be45c2b914febda0c4b58aa853320029ff5" exitCode=0
Jul 23 11:30:55.709172 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:55.708711    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z" event={"ID":"90a30124-5518-4f73-8520-95aeb0d75a20","Type":"ContainerDied","Data":"ef0b80ddfc6f0733f997304d1fde7be45c2b914febda0c4b58aa853320029ff5"}
Jul 23 11:30:56.716790 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:56.716751    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z" event={"ID":"90a30124-5518-4f73-8520-95aeb0d75a20","Type":"ContainerStarted","Data":"bc7d5fc5e34fa2c10c1fe95bbd7b72886fc32a900b6377d84ef08d5ac02ecf70"}
Jul 23 11:30:56.717204 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:56.716848    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:30:56.738639 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:30:56.738594    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z" podStartSLOduration=1.916437739 podStartE2EDuration="6.738577892s" totalImagesPullingTime="3.814756115s" totalInitContainerRuntime="1.007384038s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:30:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:30:56.737440179 +0000 UTC m=+271.022905947" watchObservedRunningTime="2026-07-23 11:30:56.738577892 +0000 UTC m=+271.024043658"
Jul 23 11:31:07.723732 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:07.723703    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-674bfcc66c-z5r7z"
Jul 23 11:31:19.019061 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.019008    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"]
Jul 23 11:31:19.022547 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.022527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.025669 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.025644    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 11:31:19.025816 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.025719    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c6f8c\""
Jul 23 11:31:19.041912 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.041883    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"]
Jul 23 11:31:19.104321 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.104290    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"]
Jul 23 11:31:19.107733 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.107698    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.113002 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.112978    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 11:31:19.113186 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.113168    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-drzrj\""
Jul 23 11:31:19.113398 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.113384    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 11:31:19.116565 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.116523    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 11:31:19.117163 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.117136    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 11:31:19.117672 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.117654    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 11:31:19.138543 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.138513    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"]
Jul 23 11:31:19.163567 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj4qj\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-kube-api-access-zj4qj\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163684    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163710    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163748 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163732    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163949 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.163949 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.163805    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.264627 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264592    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8bk\" (UniqueName: \"kubernetes.io/projected/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-kube-api-access-6p8bk\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.264627 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264632    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.264840 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264767    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.264873 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj4qj\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-kube-api-access-zj4qj\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.264910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264884    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-metrics-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.264946 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264911    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.264979 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264954    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265011 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265011 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.264995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265089 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265089 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265200 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265177    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265251 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-manager-config\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.265399 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265372    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265576 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265510    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265637 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.265776 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.266039 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.265908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-data\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.267628 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.267607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.267773 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.267756    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.282904 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.282844    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj4qj\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-kube-api-access-zj4qj\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.282904 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.282853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/539eaa16-ce0d-4526-9fcf-e08556ca3f42-istio-token\") pod \"data-science-gateway-data-science-gateway-class-656c84dccd5f8bl\" (UID: \"539eaa16-ce0d-4526-9fcf-e08556ca3f42\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.333377 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.333341    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:19.366691 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.366658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-manager-config\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.366871 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.366703    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8bk\" (UniqueName: \"kubernetes.io/projected/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-kube-api-access-6p8bk\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.366953 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.366887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.367072 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.366949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-metrics-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.367759 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.367668    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-manager-config\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.369637 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.369614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.369961 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.369942    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-metrics-cert\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.375973 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.375949    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8bk\" (UniqueName: \"kubernetes.io/projected/82e0d4cc-a02f-4faa-838b-66bb439ff7dd-kube-api-access-6p8bk\") pod \"lws-controller-manager-7c45f5478c-t5g8z\" (UID: \"82e0d4cc-a02f-4faa-838b-66bb439ff7dd\") " pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.420145 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.420110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:19.485054 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.485016    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"]
Jul 23 11:31:19.489181 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:31:19.489041    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539eaa16_ce0d_4526_9fcf_e08556ca3f42.slice/crio-4320adc2a1336f760acbf047e4a396d012ac1d6ae5cfc52e5b0cd1a1a633692f WatchSource:0}: Error finding container 4320adc2a1336f760acbf047e4a396d012ac1d6ae5cfc52e5b0cd1a1a633692f: Status 404 returned error can't find the container with id 4320adc2a1336f760acbf047e4a396d012ac1d6ae5cfc52e5b0cd1a1a633692f
Jul 23 11:31:19.568418 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.568394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"]
Jul 23 11:31:19.570387 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:31:19.570355    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e0d4cc_a02f_4faa_838b_66bb439ff7dd.slice/crio-ae4ef0f3268138111f29386b9711bf3f73f1c2cc57bbdbf54d9149d3ac8c85ef WatchSource:0}: Error finding container ae4ef0f3268138111f29386b9711bf3f73f1c2cc57bbdbf54d9149d3ac8c85ef: Status 404 returned error can't find the container with id ae4ef0f3268138111f29386b9711bf3f73f1c2cc57bbdbf54d9149d3ac8c85ef
Jul 23 11:31:19.823197 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.823103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl" event={"ID":"539eaa16-ce0d-4526-9fcf-e08556ca3f42","Type":"ContainerStarted","Data":"4320adc2a1336f760acbf047e4a396d012ac1d6ae5cfc52e5b0cd1a1a633692f"}
Jul 23 11:31:19.824300 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:19.824275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z" event={"ID":"82e0d4cc-a02f-4faa-838b-66bb439ff7dd","Type":"ContainerStarted","Data":"ae4ef0f3268138111f29386b9711bf3f73f1c2cc57bbdbf54d9149d3ac8c85ef"}
Jul 23 11:31:23.631910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.625732    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:31:23.631910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.625910    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:31:23.631910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.625993    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:31:23.848025 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.847988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl" event={"ID":"539eaa16-ce0d-4526-9fcf-e08556ca3f42","Type":"ContainerStarted","Data":"52fcbe97fb311abadef2fb27cba20dbdad3ceac1a3fc24dc6bfeb70beccdc711"}
Jul 23 11:31:23.849609 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.849585    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z" event={"ID":"82e0d4cc-a02f-4faa-838b-66bb439ff7dd","Type":"ContainerStarted","Data":"96c3174c1a56682c22940c1a58beeba503fe959ed6796fc915207cea112b43bb"}
Jul 23 11:31:23.849745 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.849639    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:31:23.880429 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.880315    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl" podStartSLOduration=1.746067938 podStartE2EDuration="5.880281928s" totalImagesPullingTime="4.13421399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:31:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:31:23.879206793 +0000 UTC m=+298.164672562" watchObservedRunningTime="2026-07-23 11:31:23.880281928 +0000 UTC m=+298.165747691"
Jul 23 11:31:23.930048 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:23.929997    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z" podStartSLOduration=0.879648078 podStartE2EDuration="4.929980885s" totalImagesPullingTime="4.050332807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:31:23.928603277 +0000 UTC m=+298.214069042" watchObservedRunningTime="2026-07-23 11:31:23.929980885 +0000 UTC m=+298.215446651"
Jul 23 11:31:24.334443 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:24.334416    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:24.339196 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:24.339173    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:24.854127 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:24.854095    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:24.855085 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:24.855067    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-656c84dccd5f8bl"
Jul 23 11:31:26.145660 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:26.145632    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:31:26.149648 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:26.149622    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:31:26.153699 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:26.153677    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:31:26.163105 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:26.163077    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 11:31:26.163274 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:26.163179    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:31:34.856081 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:31:34.856043    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c45f5478c-t5g8z"
Jul 23 11:33:08.398329 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.398295    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"]
Jul 23 11:33:08.401327 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.401312    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.405258 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.405234    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jhmwr\""
Jul 23 11:33:08.405396 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.405277    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 11:33:08.405396 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.405291    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 11:33:08.420438 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.420409    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"]
Jul 23 11:33:08.478398 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.478356    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qpmp\" (UniqueName: \"kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.478398 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.478397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.579767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.579734    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qpmp\" (UniqueName: \"kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.579902 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.579774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.580125 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.580109    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.589425 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.589400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qpmp\" (UniqueName: \"kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.711100 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.711008    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:08.852607 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.852580    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"]
Jul 23 11:33:08.855591 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:33:08.855563    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61cf38f_87c3_4b17_993b_6b32fa86e335.slice/crio-0f040e2e47cecc3c9875d1b3b9ad6be28764489e7a3206265d8081015077e801 WatchSource:0}: Error finding container 0f040e2e47cecc3c9875d1b3b9ad6be28764489e7a3206265d8081015077e801: Status 404 returned error can't find the container with id 0f040e2e47cecc3c9875d1b3b9ad6be28764489e7a3206265d8081015077e801
Jul 23 11:33:08.858103 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:08.858089    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 11:33:09.105840 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.105806    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"]
Jul 23 11:33:09.111820 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.111787    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"]
Jul 23 11:33:09.126662 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.126636    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:09.131217 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.131192    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.152963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.152930    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:09.286281 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.286232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.286281 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.286281    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br847\" (UniqueName: \"kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.387707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.387621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.387707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.387661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br847\" (UniqueName: \"kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.388076 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.388054    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.396699 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.396663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br847\" (UniqueName: \"kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-xkv28\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.441784 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.441751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:09.586326 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:09.586282    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:09.669599 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:33:09.669522    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524b38cc_643d_4ec4_903f_a1949791f51a.slice/crio-36f0ec4208da8b0d8df5a38ac2bdf5bbb98e2ebc63268119e17c0f8c66b96ac7 WatchSource:0}: Error finding container 36f0ec4208da8b0d8df5a38ac2bdf5bbb98e2ebc63268119e17c0f8c66b96ac7: Status 404 returned error can't find the container with id 36f0ec4208da8b0d8df5a38ac2bdf5bbb98e2ebc63268119e17c0f8c66b96ac7
Jul 23 11:33:10.315017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:10.314973    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" event={"ID":"524b38cc-643d-4ec4-903f-a1949791f51a","Type":"ContainerStarted","Data":"36f0ec4208da8b0d8df5a38ac2bdf5bbb98e2ebc63268119e17c0f8c66b96ac7"}
Jul 23 11:33:13.331737 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.331692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" event={"ID":"524b38cc-643d-4ec4-903f-a1949791f51a","Type":"ContainerStarted","Data":"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"}
Jul 23 11:33:13.332196 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.331767    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:13.333515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.333485    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" containerName="manager" containerID="cri-o://8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d" gracePeriod=2
Jul 23 11:33:13.356691 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.356644    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" podStartSLOduration=0.845922032 podStartE2EDuration="4.356628436s" totalImagesPullingTime="3.510706404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:33:13.353627467 +0000 UTC m=+407.639093232" watchObservedRunningTime="2026-07-23 11:33:13.356628436 +0000 UTC m=+407.642094258"
Jul 23 11:33:13.589390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.589361    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:13.592040 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:33:13.592011    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" is forbidden: User \"system:node:ip-10-0-140-183.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-183.ec2.internal' and this object" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:13.730410 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.730323    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qpmp\" (UniqueName: \"kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp\") pod \"c61cf38f-87c3-4b17-993b-6b32fa86e335\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") "
Jul 23 11:33:13.730410 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.730365    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume\") pod \"c61cf38f-87c3-4b17-993b-6b32fa86e335\" (UID: \"c61cf38f-87c3-4b17-993b-6b32fa86e335\") "
Jul 23 11:33:13.730687 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.730666    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c61cf38f-87c3-4b17-993b-6b32fa86e335" (UID: "c61cf38f-87c3-4b17-993b-6b32fa86e335"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:33:13.732740 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.732718    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp" (OuterVolumeSpecName: "kube-api-access-5qpmp") pod "c61cf38f-87c3-4b17-993b-6b32fa86e335" (UID: "c61cf38f-87c3-4b17-993b-6b32fa86e335"). InnerVolumeSpecName "kube-api-access-5qpmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:33:13.831317 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.831280    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qpmp\" (UniqueName: \"kubernetes.io/projected/c61cf38f-87c3-4b17-993b-6b32fa86e335-kube-api-access-5qpmp\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:33:13.831317 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:13.831310    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c61cf38f-87c3-4b17-993b-6b32fa86e335-extensions-socket-volume\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:33:14.298860 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.298822    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" path="/var/lib/kubelet/pods/c61cf38f-87c3-4b17-993b-6b32fa86e335/volumes"
Jul 23 11:33:14.339155 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.339117    2574 generic.go:356] "Generic (PLEG): container finished" podID="c61cf38f-87c3-4b17-993b-6b32fa86e335" containerID="8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d" exitCode=2
Jul 23 11:33:14.339662 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.339166    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:14.339662 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.339215    2574 scope.go:117] "RemoveContainer" containerID="8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d"
Jul 23 11:33:14.343883 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:33:14.343855    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" is forbidden: User \"system:node:ip-10-0-140-183.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-183.ec2.internal' and this object" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:14.348107 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.348087    2574 scope.go:117] "RemoveContainer" containerID="8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d"
Jul 23 11:33:14.348449 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:33:14.348428    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d\": container with ID starting with 8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d not found: ID does not exist" containerID="8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d"
Jul 23 11:33:14.348568 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:14.348456    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d"} err="failed to get container status \"8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d\": rpc error: code = NotFound desc = could not find container \"8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d\": container with ID starting with 8321a965c29f5704974620403eeb290ffdd8b030301718980b84604cdf010c5d not found: ID does not exist"
Jul 23 11:33:16.296540 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:33:16.296497    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-vx4q4\" is forbidden: User \"system:node:ip-10-0-140-183.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-183.ec2.internal' and this object" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-vx4q4"
Jul 23 11:33:24.341632 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:24.341597    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:36.722413 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:36.722380    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:36.722853 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:36.722623    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" podUID="524b38cc-643d-4ec4-903f-a1949791f51a" containerName="manager" containerID="cri-o://2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6" gracePeriod=10
Jul 23 11:33:36.979426 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:36.979361    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:37.010564 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.010528    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br847\" (UniqueName: \"kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847\") pod \"524b38cc-643d-4ec4-903f-a1949791f51a\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") "
Jul 23 11:33:37.010767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.010596    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume\") pod \"524b38cc-643d-4ec4-903f-a1949791f51a\" (UID: \"524b38cc-643d-4ec4-903f-a1949791f51a\") "
Jul 23 11:33:37.011006 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.010977    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "524b38cc-643d-4ec4-903f-a1949791f51a" (UID: "524b38cc-643d-4ec4-903f-a1949791f51a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:33:37.012941 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.012915    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847" (OuterVolumeSpecName: "kube-api-access-br847") pod "524b38cc-643d-4ec4-903f-a1949791f51a" (UID: "524b38cc-643d-4ec4-903f-a1949791f51a"). InnerVolumeSpecName "kube-api-access-br847". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:33:37.111897 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.111865    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br847\" (UniqueName: \"kubernetes.io/projected/524b38cc-643d-4ec4-903f-a1949791f51a-kube-api-access-br847\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:33:37.111897 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.111890    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/524b38cc-643d-4ec4-903f-a1949791f51a-extensions-socket-volume\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:33:37.447306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.447273    2574 generic.go:356] "Generic (PLEG): container finished" podID="524b38cc-643d-4ec4-903f-a1949791f51a" containerID="2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6" exitCode=0
Jul 23 11:33:37.447500 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.447322    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"
Jul 23 11:33:37.447500 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.447357    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" event={"ID":"524b38cc-643d-4ec4-903f-a1949791f51a","Type":"ContainerDied","Data":"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"}
Jul 23 11:33:37.447500 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.447413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28" event={"ID":"524b38cc-643d-4ec4-903f-a1949791f51a","Type":"ContainerDied","Data":"36f0ec4208da8b0d8df5a38ac2bdf5bbb98e2ebc63268119e17c0f8c66b96ac7"}
Jul 23 11:33:37.447500 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.447434    2574 scope.go:117] "RemoveContainer" containerID="2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"
Jul 23 11:33:37.456815 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.456801    2574 scope.go:117] "RemoveContainer" containerID="2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"
Jul 23 11:33:37.457063 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:33:37.457046    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6\": container with ID starting with 2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6 not found: ID does not exist" containerID="2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"
Jul 23 11:33:37.457108 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.457071    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6"} err="failed to get container status \"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6\": rpc error: code = NotFound desc = could not find container \"2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6\": container with ID starting with 2b0df78c34c5280aef609370b464e704427042e6a12b11c37d8c9d31691ab6c6 not found: ID does not exist"
Jul 23 11:33:37.467914 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.467887    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:37.472076 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:37.472056    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-xkv28"]
Jul 23 11:33:38.297617 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:38.297583    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524b38cc-643d-4ec4-903f-a1949791f51a" path="/var/lib/kubelet/pods/524b38cc-643d-4ec4-903f-a1949791f51a/volumes"
Jul 23 11:33:40.819503 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.819439    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"]
Jul 23 11:33:40.819962 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.819938    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" containerName="manager"
Jul 23 11:33:40.819962 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.819956    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" containerName="manager"
Jul 23 11:33:40.820064 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.819971    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="524b38cc-643d-4ec4-903f-a1949791f51a" containerName="manager"
Jul 23 11:33:40.820064 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.819981    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="524b38cc-643d-4ec4-903f-a1949791f51a" containerName="manager"
Jul 23 11:33:40.820144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.820081    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c61cf38f-87c3-4b17-993b-6b32fa86e335" containerName="manager"
Jul 23 11:33:40.820144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.820091    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="524b38cc-643d-4ec4-903f-a1949791f51a" containerName="manager"
Jul 23 11:33:40.824379 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.824360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.826864 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.826839    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hfdgn\""
Jul 23 11:33:40.833682 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.833659    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"]
Jul 23 11:33:40.841706 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.841834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.841834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.841834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841809    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.841963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841833    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rjp2\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-kube-api-access-5rjp2\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.841963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.842080 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.841967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.842080 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.842025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.842175 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.842081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35d4ad14-eb9c-4ded-8688-066b4681eed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943002 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.942969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943002 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943018    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943037    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rjp2\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-kube-api-access-5rjp2\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943090    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943185    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35d4ad14-eb9c-4ded-8688-066b4681eed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943265 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943252    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943718 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943695    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-certs\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.943833 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943808    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-credential-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.944000 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-workload-socket\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.944000 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943943    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-data\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.944113 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.943981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35d4ad14-eb9c-4ded-8688-066b4681eed8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.945932 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.945904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-envoy\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.946029 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.945908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.951495 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.951453    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-istio-token\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:40.951609 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:40.951559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rjp2\" (UniqueName: \"kubernetes.io/projected/35d4ad14-eb9c-4ded-8688-066b4681eed8-kube-api-access-5rjp2\") pod \"maas-default-gateway-openshift-default-5d7b8c9489-6zqb4\" (UID: \"35d4ad14-eb9c-4ded-8688-066b4681eed8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:41.136317 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.136275    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:41.275696 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.275671    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"]
Jul 23 11:33:41.278600 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:33:41.278572    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d4ad14_eb9c_4ded_8688_066b4681eed8.slice/crio-50cede723508db234d6dddeea4e1e5bf88feac95d43031b8634c902245d920c4 WatchSource:0}: Error finding container 50cede723508db234d6dddeea4e1e5bf88feac95d43031b8634c902245d920c4: Status 404 returned error can't find the container with id 50cede723508db234d6dddeea4e1e5bf88feac95d43031b8634c902245d920c4
Jul 23 11:33:41.280759 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.280728    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:33:41.280827 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.280804    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:33:41.280865 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.280833    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:33:41.469144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.469053    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" event={"ID":"35d4ad14-eb9c-4ded-8688-066b4681eed8","Type":"ContainerStarted","Data":"8563f4aadbb50d705ef04bbad4b0e7a46695850d887e0d21a6e6475e092fd440"}
Jul 23 11:33:41.469144 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.469102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" event={"ID":"35d4ad14-eb9c-4ded-8688-066b4681eed8","Type":"ContainerStarted","Data":"50cede723508db234d6dddeea4e1e5bf88feac95d43031b8634c902245d920c4"}
Jul 23 11:33:41.489523 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:41.489449    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podStartSLOduration=1.4894312539999999 podStartE2EDuration="1.489431254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:33:41.487649374 +0000 UTC m=+435.773115157" watchObservedRunningTime="2026-07-23 11:33:41.489431254 +0000 UTC m=+435.774897019"
Jul 23 11:33:42.137162 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:42.137116    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:42.142245 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:42.142219    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:42.474090 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:42.474007    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:42.475160 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:42.475140    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:33:55.999964 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:55.999880    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:33:56.003073 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.003052    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.005751 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.005729    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 11:33:56.005833 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.005780    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 11:33:56.006676 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.006657    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-45x6k\""
Jul 23 11:33:56.006778 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.006676    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 11:33:56.011392 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.011371    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:33:56.040432 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.040397    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:33:56.075167 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.075138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-config-file\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.075313 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.075214    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62lx\" (UniqueName: \"kubernetes.io/projected/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-kube-api-access-b62lx\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.176654 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.176617    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62lx\" (UniqueName: \"kubernetes.io/projected/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-kube-api-access-b62lx\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.176830 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.176694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-config-file\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.177279 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.177261    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-config-file\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.186848 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.186824    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62lx\" (UniqueName: \"kubernetes.io/projected/617c5e2a-6ffb-46c8-963a-4cf8d39450cd-kube-api-access-b62lx\") pod \"limitador-limitador-54d6d988d5-tncjv\" (UID: \"617c5e2a-6ffb-46c8-963a-4cf8d39450cd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.314130 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.314039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:56.470860 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.470825    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:33:56.474823 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:33:56.474790    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617c5e2a_6ffb_46c8_963a_4cf8d39450cd.slice/crio-19a29d1c26cac14d21aa43f74cad5dea3714346fc1b655c6987e8464bcc9a362 WatchSource:0}: Error finding container 19a29d1c26cac14d21aa43f74cad5dea3714346fc1b655c6987e8464bcc9a362: Status 404 returned error can't find the container with id 19a29d1c26cac14d21aa43f74cad5dea3714346fc1b655c6987e8464bcc9a362
Jul 23 11:33:56.536724 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:56.536692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv" event={"ID":"617c5e2a-6ffb-46c8-963a-4cf8d39450cd","Type":"ContainerStarted","Data":"19a29d1c26cac14d21aa43f74cad5dea3714346fc1b655c6987e8464bcc9a362"}
Jul 23 11:33:59.552017 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:59.551920    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv" event={"ID":"617c5e2a-6ffb-46c8-963a-4cf8d39450cd","Type":"ContainerStarted","Data":"d81f76c85cc796860a8724689c2d2912ec888c42f1b1442ef93681d5481679a4"}
Jul 23 11:33:59.552411 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:59.552071    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:33:59.569720 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:33:59.569672    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv" podStartSLOduration=1.957366095 podStartE2EDuration="4.569654058s" totalImagesPullingTime="2.612287963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:33:59.568333143 +0000 UTC m=+453.853798909" watchObservedRunningTime="2026-07-23 11:33:59.569654058 +0000 UTC m=+453.855119823"
Jul 23 11:34:10.557443 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:10.557413    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tncjv"
Jul 23 11:34:32.809516 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.809476    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:32.817145 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.817120    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:32.820154 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.820133    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 23 11:34:32.820276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.820150    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bxwkd\""
Jul 23 11:34:32.820446 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.820432    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 23 11:34:32.825113 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.825043    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:32.902525 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.902483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64d7v\" (UniqueName: \"kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:32.902707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.902537    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:32.994393 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.994353    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:34:32.998062 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:32.998044    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.003387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.003363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64d7v\" (UniqueName: \"kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.003536 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.003401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.003587 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:34:33.003550    2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 23 11:34:33.003632 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:34:33.003621    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert podName:f751f151-bcb2-4d5c-9649-be857dab1b69 nodeName:}" failed. No retries permitted until 2026-07-23 11:34:33.503601478 +0000 UTC m=+487.789067223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert") pod "maas-controller-b4595677c-qlhz7" (UID: "f751f151-bcb2-4d5c-9649-be857dab1b69") : secret "maas-controller-webhook-cert" not found
Jul 23 11:34:33.007777 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.007749    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:34:33.012319 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.012297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64d7v\" (UniqueName: \"kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.104304 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.104270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.104519 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.104410    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88nwl\" (UniqueName: \"kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.205359 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.205322    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88nwl\" (UniqueName: \"kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.205359 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.205364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.207999 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.207969    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.213515 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.213491    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88nwl\" (UniqueName: \"kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl\") pod \"maas-controller-79c669ccc6-gxcsh\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") " pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.309904 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.309862    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:33.447312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.447269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:34:33.450169 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:34:33.450139    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d044808_798c_41c3_a274_1f83ced54ee3.slice/crio-deb58d3f24a8eb4876a2bae108673870e5d2450ac30d8ef85ee3d242c2320195 WatchSource:0}: Error finding container deb58d3f24a8eb4876a2bae108673870e5d2450ac30d8ef85ee3d242c2320195: Status 404 returned error can't find the container with id deb58d3f24a8eb4876a2bae108673870e5d2450ac30d8ef85ee3d242c2320195
Jul 23 11:34:33.508018 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.507980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.510585 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.510554    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") pod \"maas-controller-b4595677c-qlhz7\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") " pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.702217 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.702125    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" event={"ID":"8d044808-798c-41c3-a274-1f83ced54ee3","Type":"ContainerStarted","Data":"deb58d3f24a8eb4876a2bae108673870e5d2450ac30d8ef85ee3d242c2320195"}
Jul 23 11:34:33.729657 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.729622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:33.867524 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:33.867497    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:33.869612 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:34:33.869577    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf751f151_bcb2_4d5c_9649_be857dab1b69.slice/crio-52bab248a4dada916e7d3c1ba38e2f84a5cbc69fc631063b99b2ab8715782920 WatchSource:0}: Error finding container 52bab248a4dada916e7d3c1ba38e2f84a5cbc69fc631063b99b2ab8715782920: Status 404 returned error can't find the container with id 52bab248a4dada916e7d3c1ba38e2f84a5cbc69fc631063b99b2ab8715782920
Jul 23 11:34:34.711740 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:34.711700    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4595677c-qlhz7" event={"ID":"f751f151-bcb2-4d5c-9649-be857dab1b69","Type":"ContainerStarted","Data":"52bab248a4dada916e7d3c1ba38e2f84a5cbc69fc631063b99b2ab8715782920"}
Jul 23 11:34:36.722387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.722344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" event={"ID":"8d044808-798c-41c3-a274-1f83ced54ee3","Type":"ContainerStarted","Data":"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"}
Jul 23 11:34:36.722876 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.722561    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:36.723784 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.723761    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4595677c-qlhz7" event={"ID":"f751f151-bcb2-4d5c-9649-be857dab1b69","Type":"ContainerStarted","Data":"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"}
Jul 23 11:34:36.723887 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.723853    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:36.756107 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.756052    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" podStartSLOduration=1.922425273 podStartE2EDuration="4.756036823s" totalImagesPullingTime="2.83361155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:34:36.754872478 +0000 UTC m=+491.040338247" watchObservedRunningTime="2026-07-23 11:34:36.756036823 +0000 UTC m=+491.041502607"
Jul 23 11:34:36.786574 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:36.786524    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b4595677c-qlhz7" podStartSLOduration=2.382301413 podStartE2EDuration="4.786509113s" totalImagesPullingTime="2.4042077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:34:36.784692438 +0000 UTC m=+491.070158215" watchObservedRunningTime="2026-07-23 11:34:36.786509113 +0000 UTC m=+491.071974878"
Jul 23 11:34:47.731566 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:47.731536    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:47.732076 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:47.731695    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:34:47.791286 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:47.791253    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:47.791905 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:47.791618    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b4595677c-qlhz7" podUID="f751f151-bcb2-4d5c-9649-be857dab1b69" containerName="manager" containerID="cri-o://666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69" gracePeriod=30
Jul 23 11:34:48.052893 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.052868    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:48.142120 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.142080    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") pod \"f751f151-bcb2-4d5c-9649-be857dab1b69\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") "
Jul 23 11:34:48.142316 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.142226    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64d7v\" (UniqueName: \"kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v\") pod \"f751f151-bcb2-4d5c-9649-be857dab1b69\" (UID: \"f751f151-bcb2-4d5c-9649-be857dab1b69\") "
Jul 23 11:34:48.144591 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.144561    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v" (OuterVolumeSpecName: "kube-api-access-64d7v") pod "f751f151-bcb2-4d5c-9649-be857dab1b69" (UID: "f751f151-bcb2-4d5c-9649-be857dab1b69"). InnerVolumeSpecName "kube-api-access-64d7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:34:48.144591 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.144577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f751f151-bcb2-4d5c-9649-be857dab1b69" (UID: "f751f151-bcb2-4d5c-9649-be857dab1b69"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:34:48.243574 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.243532    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64d7v\" (UniqueName: \"kubernetes.io/projected/f751f151-bcb2-4d5c-9649-be857dab1b69-kube-api-access-64d7v\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:34:48.243574 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.243567    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f751f151-bcb2-4d5c-9649-be857dab1b69-webhook-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:34:48.781415 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.781377    2574 generic.go:356] "Generic (PLEG): container finished" podID="f751f151-bcb2-4d5c-9649-be857dab1b69" containerID="666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69" exitCode=0
Jul 23 11:34:48.781943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.781423    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b4595677c-qlhz7"
Jul 23 11:34:48.781943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.781480    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4595677c-qlhz7" event={"ID":"f751f151-bcb2-4d5c-9649-be857dab1b69","Type":"ContainerDied","Data":"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"}
Jul 23 11:34:48.781943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.781546    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b4595677c-qlhz7" event={"ID":"f751f151-bcb2-4d5c-9649-be857dab1b69","Type":"ContainerDied","Data":"52bab248a4dada916e7d3c1ba38e2f84a5cbc69fc631063b99b2ab8715782920"}
Jul 23 11:34:48.781943 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.781579    2574 scope.go:117] "RemoveContainer" containerID="666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"
Jul 23 11:34:48.790127 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.790071    2574 scope.go:117] "RemoveContainer" containerID="666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"
Jul 23 11:34:48.790348 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:34:48.790330    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69\": container with ID starting with 666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69 not found: ID does not exist" containerID="666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"
Jul 23 11:34:48.790409 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.790360    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69"} err="failed to get container status \"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69\": rpc error: code = NotFound desc = could not find container \"666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69\": container with ID starting with 666b826bd1cfffc946997f60a64d368369a26ea60cc11088b6de9160b8903b69 not found: ID does not exist"
Jul 23 11:34:48.797126 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.797105    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:48.800503 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:48.800483    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b4595677c-qlhz7"]
Jul 23 11:34:50.297122 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:34:50.297089    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f751f151-bcb2-4d5c-9649-be857dab1b69" path="/var/lib/kubelet/pods/f751f151-bcb2-4d5c-9649-be857dab1b69/volumes"
Jul 23 11:35:12.442987 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.442949    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"]
Jul 23 11:35:12.443510 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.443489    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f751f151-bcb2-4d5c-9649-be857dab1b69" containerName="manager"
Jul 23 11:35:12.443593 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.443513    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f751f151-bcb2-4d5c-9649-be857dab1b69" containerName="manager"
Jul 23 11:35:12.443644 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.443615    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f751f151-bcb2-4d5c-9649-be857dab1b69" containerName="manager"
Jul 23 11:35:12.450111 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.450088    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.453128 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.453087    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 11:35:12.453128 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.453091    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 23 11:35:12.453318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.453169    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 11:35:12.454134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.454118    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wwzjm\""
Jul 23 11:35:12.459742 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.459716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"]
Jul 23 11:35:12.564599 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.564567    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff04c43-1c40-4272-97f6-61adc3ca3b9e-openshift-service-ca\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.564599 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.564604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df9zf\" (UniqueName: \"kubernetes.io/projected/dff04c43-1c40-4272-97f6-61adc3ca3b9e-kube-api-access-df9zf\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.564821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.564638    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.666266 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.666215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff04c43-1c40-4272-97f6-61adc3ca3b9e-openshift-service-ca\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.666266 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.666266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df9zf\" (UniqueName: \"kubernetes.io/projected/dff04c43-1c40-4272-97f6-61adc3ca3b9e-kube-api-access-df9zf\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.666585 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.666312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.666585 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:35:12.666512    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 23 11:35:12.666694 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:35:12.666592    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls podName:dff04c43-1c40-4272-97f6-61adc3ca3b9e nodeName:}" failed. No retries permitted until 2026-07-23 11:35:13.166570755 +0000 UTC m=+527.452036506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls") pod "maas-api-6779586b89-d9pbz" (UID: "dff04c43-1c40-4272-97f6-61adc3ca3b9e") : secret "maas-api-serving-cert" not found
Jul 23 11:35:12.667021 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.666997    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dff04c43-1c40-4272-97f6-61adc3ca3b9e-openshift-service-ca\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:12.677060 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:12.677032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df9zf\" (UniqueName: \"kubernetes.io/projected/dff04c43-1c40-4272-97f6-61adc3ca3b9e-kube-api-access-df9zf\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:13.120716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.120680    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:35:13.169961 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.169925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:13.173542 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.173362    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dff04c43-1c40-4272-97f6-61adc3ca3b9e-maas-api-tls\") pod \"maas-api-6779586b89-d9pbz\" (UID: \"dff04c43-1c40-4272-97f6-61adc3ca3b9e\") " pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:13.361795 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.361759    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:13.503097 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.503065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"]
Jul 23 11:35:13.506891 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:35:13.506863    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff04c43_1c40_4272_97f6_61adc3ca3b9e.slice/crio-8ca0d2143bd8b68192c62abbda9fdb8c81f7523eae1818a8bba0a01e94346338 WatchSource:0}: Error finding container 8ca0d2143bd8b68192c62abbda9fdb8c81f7523eae1818a8bba0a01e94346338: Status 404 returned error can't find the container with id 8ca0d2143bd8b68192c62abbda9fdb8c81f7523eae1818a8bba0a01e94346338
Jul 23 11:35:13.903253 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:13.903165    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz" event={"ID":"dff04c43-1c40-4272-97f6-61adc3ca3b9e","Type":"ContainerStarted","Data":"8ca0d2143bd8b68192c62abbda9fdb8c81f7523eae1818a8bba0a01e94346338"}
Jul 23 11:35:15.914619 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:15.914582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz" event={"ID":"dff04c43-1c40-4272-97f6-61adc3ca3b9e","Type":"ContainerStarted","Data":"a4eaa2e62f17fea09071a2d13f038c43168c53abcdd56514395c0613e6c90468"}
Jul 23 11:35:15.915034 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:15.914708    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:15.933070 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:15.933020    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz" podStartSLOduration=2.237945546 podStartE2EDuration="3.933002331s" totalImagesPullingTime="1.695056785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:35:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:35:15.931492997 +0000 UTC m=+530.216958776" watchObservedRunningTime="2026-07-23 11:35:15.933002331 +0000 UTC m=+530.218468100"
Jul 23 11:35:20.939053 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.938960    2574 generic.go:356] "Generic (PLEG): container finished" podID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerID="8563f4aadbb50d705ef04bbad4b0e7a46695850d887e0d21a6e6475e092fd440" exitCode=0
Jul 23 11:35:20.939053 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.939032    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" event={"ID":"35d4ad14-eb9c-4ded-8688-066b4681eed8","Type":"ContainerDied","Data":"8563f4aadbb50d705ef04bbad4b0e7a46695850d887e0d21a6e6475e092fd440"}
Jul 23 11:35:20.939501 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.939424    2574 scope.go:117] "RemoveContainer" containerID="8563f4aadbb50d705ef04bbad4b0e7a46695850d887e0d21a6e6475e092fd440"
Jul 23 11:35:20.940432 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.940407    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:35:20.940518 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.940487    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:35:20.940558 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:20.940518    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:35:21.924347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:21.924318    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6779586b89-d9pbz"
Jul 23 11:35:21.946168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:21.946133    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" event={"ID":"35d4ad14-eb9c-4ded-8688-066b4681eed8","Type":"ContainerStarted","Data":"d1b85e5d9a1663f8ba5d0b389aaf72721404739eee85ff371d44fb5f1f812dc2"}
Jul 23 11:35:22.136820 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:22.136781    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:35:22.137071 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:22.137050    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 23 11:35:22.137129 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:22.137110    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podUID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 23 11:35:23.137316 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:23.137279    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 23 11:35:23.137753 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:23.137343    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podUID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 23 11:35:24.137685 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:24.137651    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 23 11:35:24.138075 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:24.137740    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podUID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 23 11:35:25.136958 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:25.136921    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 23 11:35:25.137143 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:25.136982    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podUID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 23 11:35:26.137078 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:26.137039    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 23 11:35:26.137454 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:26.137110    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4" podUID="35d4ad14-eb9c-4ded-8688-066b4681eed8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 23 11:35:27.141179 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:27.141145    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:35:27.141616 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:27.141498    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:35:27.142244 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:27.142226    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5d7b8c9489-6zqb4"
Jul 23 11:35:32.645414 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.645374    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"]
Jul 23 11:35:32.648572 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.648551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.651150 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.651088    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 11:35:32.651261 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.651123    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 11:35:32.652333 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.652313    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 23 11:35:32.652399 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.652316    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q2mvb\""
Jul 23 11:35:32.657930 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.657907    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"]
Jul 23 11:35:32.792207 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792174    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.792381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.792381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.792381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792312    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7gmk\" (UniqueName: \"kubernetes.io/projected/ce15ab8d-f2ca-4d43-9542-5c22af07a931-kube-api-access-z7gmk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.792381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.792381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.792374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893631 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893641    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7gmk\" (UniqueName: \"kubernetes.io/projected/ce15ab8d-f2ca-4d43-9542-5c22af07a931-kube-api-access-z7gmk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893729    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.893829 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.893762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.894096 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.894073    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.894161 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.894093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.894210 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.894156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.896327 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.896271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce15ab8d-f2ca-4d43-9542-5c22af07a931-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.896576 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.896560    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce15ab8d-f2ca-4d43-9542-5c22af07a931-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.904451 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.904426    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7gmk\" (UniqueName: \"kubernetes.io/projected/ce15ab8d-f2ca-4d43-9542-5c22af07a931-kube-api-access-z7gmk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn\" (UID: \"ce15ab8d-f2ca-4d43-9542-5c22af07a931\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:32.959309 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:32.959267    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:33.115831 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:33.115806    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"]
Jul 23 11:35:33.118208 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:35:33.118181    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce15ab8d_f2ca_4d43_9542_5c22af07a931.slice/crio-236d843e8c57501e287ec5bd4326e7c68cd875bbb8cb2be619405b58fd80538f WatchSource:0}: Error finding container 236d843e8c57501e287ec5bd4326e7c68cd875bbb8cb2be619405b58fd80538f: Status 404 returned error can't find the container with id 236d843e8c57501e287ec5bd4326e7c68cd875bbb8cb2be619405b58fd80538f
Jul 23 11:35:33.232423 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:33.232324    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:35:34.008346 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:34.008306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn" event={"ID":"ce15ab8d-f2ca-4d43-9542-5c22af07a931","Type":"ContainerStarted","Data":"236d843e8c57501e287ec5bd4326e7c68cd875bbb8cb2be619405b58fd80538f"}
Jul 23 11:35:36.019760 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:36.019667    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn" event={"ID":"ce15ab8d-f2ca-4d43-9542-5c22af07a931","Type":"ContainerStarted","Data":"1d74c80bf9afdffaa7c92db6d91d1e83a1c16c8600974b877c575ebca2a9b292"}
Jul 23 11:35:36.037121 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:36.037060    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn" podStartSLOduration=1.495115856 podStartE2EDuration="4.037043325s" totalImagesPullingTime="2.541927469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:35:36.03592082 +0000 UTC m=+550.321386600" watchObservedRunningTime="2026-07-23 11:35:36.037043325 +0000 UTC m=+550.322509098"
Jul 23 11:35:41.337901 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:41.337864    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:35:42.960041 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:42.960006    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:42.960449 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:42.960115    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:42.972546 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:42.972521    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:43.063306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:43.063279    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn"
Jul 23 11:35:48.529927 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:35:48.529892    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:36:01.133409 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:01.133378    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:36:06.234705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.234668    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"]
Jul 23 11:36:06.238325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.238301    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.240903 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.240881    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 11:36:06.249092 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.249062    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"]
Jul 23 11:36:06.305972 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.305802    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.305972 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.305868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.305972 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.305945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k8tf\" (UniqueName: \"kubernetes.io/projected/6f0cee02-745f-47b3-90d9-4b635b0b9ade-kube-api-access-8k8tf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.306272 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.305986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.306272 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.306017    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.306272 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.306140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406742 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k8tf\" (UniqueName: \"kubernetes.io/projected/6f0cee02-745f-47b3-90d9-4b635b0b9ade-kube-api-access-8k8tf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406832    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.406963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.406857    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.407261 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.407217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.407324 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.407244    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.407387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.407327    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.409365 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.409341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f0cee02-745f-47b3-90d9-4b635b0b9ade-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.409589 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.409561    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f0cee02-745f-47b3-90d9-4b635b0b9ade-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.416315 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.416292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k8tf\" (UniqueName: \"kubernetes.io/projected/6f0cee02-745f-47b3-90d9-4b635b0b9ade-kube-api-access-8k8tf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ls6w5\" (UID: \"6f0cee02-745f-47b3-90d9-4b635b0b9ade\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.550620 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.550541    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:06.690987 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:06.690814    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"]
Jul 23 11:36:06.693424 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:36:06.693389    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0cee02_745f_47b3_90d9_4b635b0b9ade.slice/crio-da1b34a7e785a2c0c1a54293d54e1dfcac94e6b9b6ec317819fd0d067a8660ba WatchSource:0}: Error finding container da1b34a7e785a2c0c1a54293d54e1dfcac94e6b9b6ec317819fd0d067a8660ba: Status 404 returned error can't find the container with id da1b34a7e785a2c0c1a54293d54e1dfcac94e6b9b6ec317819fd0d067a8660ba
Jul 23 11:36:07.163303 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:07.163207    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5" event={"ID":"6f0cee02-745f-47b3-90d9-4b635b0b9ade","Type":"ContainerStarted","Data":"f4c067c1995fbe3aa9bc5d0fad1825c334e8767b2e15968dbe87a1b11f2aef17"}
Jul 23 11:36:07.163303 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:07.163264    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5" event={"ID":"6f0cee02-745f-47b3-90d9-4b635b0b9ade","Type":"ContainerStarted","Data":"da1b34a7e785a2c0c1a54293d54e1dfcac94e6b9b6ec317819fd0d067a8660ba"}
Jul 23 11:36:07.181731 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:07.181676    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5" podStartSLOduration=0.987394209 podStartE2EDuration="1.181658203s" totalImagesPullingTime="194.263994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:36:07.179981341 +0000 UTC m=+581.465447106" watchObservedRunningTime="2026-07-23 11:36:07.181658203 +0000 UTC m=+581.467123947"
Jul 23 11:36:09.335076 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:09.335039    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:36:16.551357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:16.551316    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:16.551762 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:16.551386    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:16.564326 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:16.564297    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:17.222887 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:17.222859    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ls6w5"
Jul 23 11:36:18.129409 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.129377    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"]
Jul 23 11:36:18.132869 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.132840    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.135535 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.135513    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 11:36:18.142020 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.141992    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"]
Jul 23 11:36:18.217260 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.217434 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403350ec-fc63-4224-ad1a-86a80749dc58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.217434 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s799f\" (UniqueName: \"kubernetes.io/projected/403350ec-fc63-4224-ad1a-86a80749dc58-kube-api-access-s799f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.217434 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217399    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.217596 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217448    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.217596 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.217523    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.318825 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.318782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403350ec-fc63-4224-ad1a-86a80749dc58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319023 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.318833    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s799f\" (UniqueName: \"kubernetes.io/projected/403350ec-fc63-4224-ad1a-86a80749dc58-kube-api-access-s799f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319023 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.318883    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319287    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319357 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319501    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319771 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319581    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.319881 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.319817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.321987 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.321947    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403350ec-fc63-4224-ad1a-86a80749dc58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.322131 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.322103    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403350ec-fc63-4224-ad1a-86a80749dc58-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.327318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.327296    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s799f\" (UniqueName: \"kubernetes.io/projected/403350ec-fc63-4224-ad1a-86a80749dc58-kube-api-access-s799f\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gwb86\" (UID: \"403350ec-fc63-4224-ad1a-86a80749dc58\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.444122 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.444033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:18.583878 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:18.583852    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"]
Jul 23 11:36:18.585800 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:36:18.585771    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod403350ec_fc63_4224_ad1a_86a80749dc58.slice/crio-bff67ed5725fdefd830a984222d74fd48c88fe962736486bb46d2d0db316e4b7 WatchSource:0}: Error finding container bff67ed5725fdefd830a984222d74fd48c88fe962736486bb46d2d0db316e4b7: Status 404 returned error can't find the container with id bff67ed5725fdefd830a984222d74fd48c88fe962736486bb46d2d0db316e4b7
Jul 23 11:36:19.220769 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:19.220733    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86" event={"ID":"403350ec-fc63-4224-ad1a-86a80749dc58","Type":"ContainerStarted","Data":"90b39d6dc8d8ae5f931e2be423574b0368211f1c6ffc593407d8389cb2dfeb9f"}
Jul 23 11:36:19.220769 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:19.220775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86" event={"ID":"403350ec-fc63-4224-ad1a-86a80749dc58","Type":"ContainerStarted","Data":"bff67ed5725fdefd830a984222d74fd48c88fe962736486bb46d2d0db316e4b7"}
Jul 23 11:36:19.240989 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:19.240935    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86" podStartSLOduration=1.010268787 podStartE2EDuration="1.240918756s" totalImagesPullingTime="230.649969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:36:19.238518629 +0000 UTC m=+593.523984392" watchObservedRunningTime="2026-07-23 11:36:19.240918756 +0000 UTC m=+593.526384520"
Jul 23 11:36:21.026898 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:21.026858    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:36:26.182035 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:26.182001    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:36:26.185221 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:26.185197    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:36:26.188399 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:26.188378    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:36:26.191989 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:26.191970    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:36:28.444974 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:28.444931    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:28.444974 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:28.444985    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:28.457758 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:28.457731    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:36:29.285236 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:36:29.285205    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gwb86"
Jul 23 11:38:03.972194 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:03.972159    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:38:13.525340 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:13.525303    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:38:21.173501 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.173398    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:38:21.174012 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.173672    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" podUID="8d044808-798c-41c3-a274-1f83ced54ee3" containerName="manager" containerID="cri-o://ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6" gracePeriod=30
Jul 23 11:38:21.424423 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.424367    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:38:21.557211 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.557175    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88nwl\" (UniqueName: \"kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl\") pod \"8d044808-798c-41c3-a274-1f83ced54ee3\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") "
Jul 23 11:38:21.557400 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.557279    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert\") pod \"8d044808-798c-41c3-a274-1f83ced54ee3\" (UID: \"8d044808-798c-41c3-a274-1f83ced54ee3\") "
Jul 23 11:38:21.559500 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.559447    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8d044808-798c-41c3-a274-1f83ced54ee3" (UID: "8d044808-798c-41c3-a274-1f83ced54ee3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:38:21.559600 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.559445    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl" (OuterVolumeSpecName: "kube-api-access-88nwl") pod "8d044808-798c-41c3-a274-1f83ced54ee3" (UID: "8d044808-798c-41c3-a274-1f83ced54ee3"). InnerVolumeSpecName "kube-api-access-88nwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:38:21.658489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.658434    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8d044808-798c-41c3-a274-1f83ced54ee3-webhook-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:38:21.658489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.658488    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88nwl\" (UniqueName: \"kubernetes.io/projected/8d044808-798c-41c3-a274-1f83ced54ee3-kube-api-access-88nwl\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:38:21.800088 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.800001    2574 generic.go:356] "Generic (PLEG): container finished" podID="8d044808-798c-41c3-a274-1f83ced54ee3" containerID="ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6" exitCode=0
Jul 23 11:38:21.800088 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.800049    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79c669ccc6-gxcsh"
Jul 23 11:38:21.800259 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.800087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" event={"ID":"8d044808-798c-41c3-a274-1f83ced54ee3","Type":"ContainerDied","Data":"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"}
Jul 23 11:38:21.800259 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.800143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79c669ccc6-gxcsh" event={"ID":"8d044808-798c-41c3-a274-1f83ced54ee3","Type":"ContainerDied","Data":"deb58d3f24a8eb4876a2bae108673870e5d2450ac30d8ef85ee3d242c2320195"}
Jul 23 11:38:21.800259 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.800165    2574 scope.go:117] "RemoveContainer" containerID="ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"
Jul 23 11:38:21.809661 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.809638    2574 scope.go:117] "RemoveContainer" containerID="ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"
Jul 23 11:38:21.810007 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:38:21.809984    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6\": container with ID starting with ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6 not found: ID does not exist" containerID="ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"
Jul 23 11:38:21.810081 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.810020    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6"} err="failed to get container status \"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6\": rpc error: code = NotFound desc = could not find container \"ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6\": container with ID starting with ccc36f0738fc7a1e652e46aec74910ba1d61f064f38b300ac18ce63dba4a56b6 not found: ID does not exist"
Jul 23 11:38:21.821231 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.821191    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:38:21.825328 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:21.825304    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79c669ccc6-gxcsh"]
Jul 23 11:38:22.297323 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:22.297291    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d044808-798c-41c3-a274-1f83ced54ee3" path="/var/lib/kubelet/pods/8d044808-798c-41c3-a274-1f83ced54ee3/volumes"
Jul 23 11:38:22.939545 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:38:22.939500    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:40:09.226200 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:40:09.226168    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:40:18.540703 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:40:18.540664    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:40:46.440055 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:40:46.440017    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:40:56.532226 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:40:56.532190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:05.132624 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:05.132585    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:16.435526 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:16.435489    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:26.211902 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:26.211808    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:41:26.212715 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:26.212623    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:41:26.218732 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:26.218712    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:41:26.219118 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:26.219099    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:41:26.332331 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:26.332301    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:42.736522 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:42.736482    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:48.930256 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:48.930220    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:41:56.626809 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:41:56.626774    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:42:19.229478 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:42:19.229430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:42:28.329671 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:42:28.329634    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:42:50.945016 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:42:50.944926    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:03.029529 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:03.029490    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:18.030381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:18.030347    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:27.737445 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:27.737413    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:37.432261 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:37.432221    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:46.730115 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:46.730079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:43:55.338853 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:43:55.338816    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:45:45.623797 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:45:45.623724    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:45:57.036063 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:45:57.035986    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:05.855208 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:05.855168    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:18.337146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:18.337110    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:26.244651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:26.244617    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:46:26.249944 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:26.249921    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:46:26.251245 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:26.251214    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:46:26.256690 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:26.256671    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:46:30.125846 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:30.125813    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:40.830249 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:40.830215    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:48.736315 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:48.736281    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:46:59.726654 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:46:59.726618    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:47:10.832455 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:47:10.832413    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:47:20.233235 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:47:20.233153    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:47:30.949213 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:47:30.949176    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:47:57.526885 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:47:57.526846    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:48:04.937901 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:04.937864    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:48:05.139069 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.139036    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:05.139409 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.139396    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d044808-798c-41c3-a274-1f83ced54ee3" containerName="manager"
Jul 23 11:48:05.139507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.139411    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d044808-798c-41c3-a274-1f83ced54ee3" containerName="manager"
Jul 23 11:48:05.139507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.139500    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d044808-798c-41c3-a274-1f83ced54ee3" containerName="manager"
Jul 23 11:48:05.142508 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.142488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.145310 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.145286    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-694b89c1-kserve-self-signed-certs\""
Jul 23 11:48:05.161267 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.161239    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:05.195633 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195552    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.195633 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.195825 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhbch\" (UniqueName: \"kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.195825 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.195825 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195804    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.195923 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.195831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297168 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhbch\" (UniqueName: \"kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297597 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297524    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.297667 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.297633    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.298116 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.298063    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.298299 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.298219    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.300306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.300275    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.300440 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.300300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.305327 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.305300    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhbch\" (UniqueName: \"kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch\") pod \"e2e-same-id-a-694b89c1-kserve-95c88576c-l5256\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") " pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.452040 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.451936    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:05.598281 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:05.598244    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:05.602934 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:48:05.602902    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod933931a9_c37b_48fe_94f1_1c66576b3839.slice/crio-3126750b88728eacd7b6edb137df45153315de64a6c2b52681fa4d85e5c0d448 WatchSource:0}: Error finding container 3126750b88728eacd7b6edb137df45153315de64a6c2b52681fa4d85e5c0d448: Status 404 returned error can't find the container with id 3126750b88728eacd7b6edb137df45153315de64a6c2b52681fa4d85e5c0d448
Jul 23 11:48:06.523965 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:06.523929    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" event={"ID":"933931a9-c37b-48fe-94f1-1c66576b3839","Type":"ContainerStarted","Data":"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"}
Jul 23 11:48:06.524373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:06.523975    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" event={"ID":"933931a9-c37b-48fe-94f1-1c66576b3839","Type":"ContainerStarted","Data":"3126750b88728eacd7b6edb137df45153315de64a6c2b52681fa4d85e5c0d448"}
Jul 23 11:48:06.541749 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:06.541695    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" podStartSLOduration=1.5416782869999999 podStartE2EDuration="1.541678287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:48:06.540840335 +0000 UTC m=+1300.826306103" watchObservedRunningTime="2026-07-23 11:48:06.541678287 +0000 UTC m=+1300.827144052"
Jul 23 11:48:15.452684 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:15.452596    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:15.452684 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:15.452647    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:15.465482 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:15.465445    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:15.576096 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:15.576068    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:17.634946 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:17.634910    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:48:36.914213 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:36.914181    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:41.129991 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:41.129956    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:48:51.935889 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:51.935798    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" podUID="933931a9-c37b-48fe-94f1-1c66576b3839" containerName="main" containerID="cri-o://863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c" gracePeriod=45
Jul 23 11:48:52.188448 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.188393    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:52.298511 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298454    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhbch\" (UniqueName: \"kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298566    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298585    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298601    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298638    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298676    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home\") pod \"933931a9-c37b-48fe-94f1-1c66576b3839\" (UID: \"933931a9-c37b-48fe-94f1-1c66576b3839\") "
Jul 23 11:48:52.298935 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298896    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:48:52.298992 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298922    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache" (OuterVolumeSpecName: "model-cache") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:48:52.299055 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.298985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home" (OuterVolumeSpecName: "home") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:48:52.299184 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.299168    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.299253 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.299186    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.299253 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.299194    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.300914 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.300890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch" (OuterVolumeSpecName: "kube-api-access-nhbch") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "kube-api-access-nhbch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:48:52.301014 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.300923    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:48:52.301014 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.300985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm" (OuterVolumeSpecName: "dshm") pod "933931a9-c37b-48fe-94f1-1c66576b3839" (UID: "933931a9-c37b-48fe-94f1-1c66576b3839"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:48:52.400446 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.400415    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhbch\" (UniqueName: \"kubernetes.io/projected/933931a9-c37b-48fe-94f1-1c66576b3839-kube-api-access-nhbch\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.400446 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.400443    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933931a9-c37b-48fe-94f1-1c66576b3839-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.400446 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.400454    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933931a9-c37b-48fe-94f1-1c66576b3839-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:48:52.742257 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.742216    2574 generic.go:356] "Generic (PLEG): container finished" podID="933931a9-c37b-48fe-94f1-1c66576b3839" containerID="863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c" exitCode=0
Jul 23 11:48:52.742257 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.742261    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"
Jul 23 11:48:52.742451 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.742307    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" event={"ID":"933931a9-c37b-48fe-94f1-1c66576b3839","Type":"ContainerDied","Data":"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"}
Jul 23 11:48:52.742451 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.742350    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256" event={"ID":"933931a9-c37b-48fe-94f1-1c66576b3839","Type":"ContainerDied","Data":"3126750b88728eacd7b6edb137df45153315de64a6c2b52681fa4d85e5c0d448"}
Jul 23 11:48:52.742451 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.742373    2574 scope.go:117] "RemoveContainer" containerID="863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"
Jul 23 11:48:52.752177 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.752151    2574 scope.go:117] "RemoveContainer" containerID="863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"
Jul 23 11:48:52.752430 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:48:52.752412    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c\": container with ID starting with 863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c not found: ID does not exist" containerID="863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"
Jul 23 11:48:52.752504 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.752440    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c"} err="failed to get container status \"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c\": rpc error: code = NotFound desc = could not find container \"863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c\": container with ID starting with 863f8272f5eb4855d20aae789de2bd38f11f67c49bbb44c16b4d77760866856c not found: ID does not exist"
Jul 23 11:48:52.763244 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.763221    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:52.767385 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:52.767367    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-694b89c1-kserve-95c88576c-l5256"]
Jul 23 11:48:54.298273 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:54.298240    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933931a9-c37b-48fe-94f1-1c66576b3839" path="/var/lib/kubelet/pods/933931a9-c37b-48fe-94f1-1c66576b3839/volumes"
Jul 23 11:48:55.528189 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:48:55.528155    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:05.833816 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:05.833783    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:18.032200 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:18.032165    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:29.626579 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:29.626549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:38.629963 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:38.629926    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:48.951071 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:48.951034    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:49:58.338852 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:49:58.338817    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:50:07.529127 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:50:07.529096    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:50:20.830221 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:50:20.830132    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:50:30.636163 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:50:30.636126    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:50:40.034620 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:50:40.034578    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:50:50.833504 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:50:50.833452    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:03.740146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:03.740108    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:12.937187 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:12.937147    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:25.129680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:25.129634    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:26.273064 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:26.273034    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:51:26.279347 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:26.279324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:51:26.279859 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:26.279841    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:51:26.292549 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:26.292527    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:51:37.533932 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:37.533899    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:50.427119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:50.427032    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:51:59.624128 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:51:59.624092    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:52:20.830485 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:20.830432    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:52:26.693098 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.693066    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:52:26.693590 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.693489    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933931a9-c37b-48fe-94f1-1c66576b3839" containerName="main"
Jul 23 11:52:26.693590 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.693502    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="933931a9-c37b-48fe-94f1-1c66576b3839" containerName="main"
Jul 23 11:52:26.693590 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.693590    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="933931a9-c37b-48fe-94f1-1c66576b3839" containerName="main"
Jul 23 11:52:26.696580 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.696563    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.699208 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.699183    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2a4ab95e-openshift-default-dockercfg-wvn5n\""
Jul 23 11:52:26.717593 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.717568    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:52:26.820965 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.820924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821139 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.820979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821139 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821010    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76pqs\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821139 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821139 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821101    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821139 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821297 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821297 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.821297 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.821288    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.922859 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.922820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923036 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.922924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923036 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.922955    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923036 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.922978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76pqs\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923036 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923254 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923352 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923352 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923455 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923389    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923455 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923455 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923417    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923686 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923663    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923844 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923745    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.923938 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.923913    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.925929 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.925899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.926010 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.925979    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.931373 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.931347    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:26.931446 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:26.931353    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76pqs\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs\") pod \"e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") " pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:27.007734 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.007642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:27.151087 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.151065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:52:27.153113 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:52:27.153084    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82236d2_4405_4e7f_ad70_9b485b9bb8c6.slice/crio-3838fa2742f20d6bb73968cd5faf1013a77fad257c6921008ca19660925469c0 WatchSource:0}: Error finding container 3838fa2742f20d6bb73968cd5faf1013a77fad257c6921008ca19660925469c0: Status 404 returned error can't find the container with id 3838fa2742f20d6bb73968cd5faf1013a77fad257c6921008ca19660925469c0
Jul 23 11:52:27.156063 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.156030    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:27.156145 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.156103    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:27.156183 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.156156    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:27.745405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.745366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerStarted","Data":"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"}
Jul 23 11:52:27.745929 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.745448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerStarted","Data":"3838fa2742f20d6bb73968cd5faf1013a77fad257c6921008ca19660925469c0"}
Jul 23 11:52:27.784612 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:27.784555    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podStartSLOduration=1.784538766 podStartE2EDuration="1.784538766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:52:27.783862593 +0000 UTC m=+1562.069328358" watchObservedRunningTime="2026-07-23 11:52:27.784538766 +0000 UTC m=+1562.070004531"
Jul 23 11:52:28.008708 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:28.008610    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:28.013659 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:28.013626    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:28.750950 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:28.750921    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:28.751878 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:28.751859    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:29.337928 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.337894    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:52:29.341338 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.341321    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.343787 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.343762    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2a4ab95e-serving-cert\""
Jul 23 11:52:29.352983 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.352951    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:52:29.452564 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.452527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.452774 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.452583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.452774 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.452684    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpfzs\" (UniqueName: \"kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.553824 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.553786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.554008 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.553835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.554008 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.553875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpfzs\" (UniqueName: \"kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.554119 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:52:29.554012    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-serving-cert: secret "maas-api-e2e-ait-2a4ab95e-serving-cert" not found
Jul 23 11:52:29.554119 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:52:29.554090    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls podName:22787dd8-7b63-4f13-ae6b-e5b032256469 nodeName:}" failed. No retries permitted until 2026-07-23 11:52:30.054066595 +0000 UTC m=+1564.339532341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls") pod "maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" (UID: "22787dd8-7b63-4f13-ae6b-e5b032256469") : secret "maas-api-e2e-ait-2a4ab95e-serving-cert" not found
Jul 23 11:52:29.554561 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.554544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:29.562650 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:29.562629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpfzs\" (UniqueName: \"kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:30.059432 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.059390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:30.061954 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.061930    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") pod \"maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:30.253512 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.253447    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:30.389608 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.389581    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:52:30.391921 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:52:30.391894    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22787dd8_7b63_4f13_ae6b_e5b032256469.slice/crio-d5f38f6beb8dae1d7dd655601a89781dfd714e57d06337ea84263e85ee6b35c8 WatchSource:0}: Error finding container d5f38f6beb8dae1d7dd655601a89781dfd714e57d06337ea84263e85ee6b35c8: Status 404 returned error can't find the container with id d5f38f6beb8dae1d7dd655601a89781dfd714e57d06337ea84263e85ee6b35c8
Jul 23 11:52:30.393388 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.393365    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 11:52:30.761489 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:30.761394    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" event={"ID":"22787dd8-7b63-4f13-ae6b-e5b032256469","Type":"ContainerStarted","Data":"d5f38f6beb8dae1d7dd655601a89781dfd714e57d06337ea84263e85ee6b35c8"}
Jul 23 11:52:31.768998 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:31.768956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" event={"ID":"22787dd8-7b63-4f13-ae6b-e5b032256469","Type":"ContainerStarted","Data":"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"}
Jul 23 11:52:31.769390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:31.769146    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:31.787894 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:31.787853    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" podStartSLOduration=2.281045914 podStartE2EDuration="2.787836826s" totalImagesPullingTime="506.790912ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:52:31.786429439 +0000 UTC m=+1566.071895205" watchObservedRunningTime="2026-07-23 11:52:31.787836826 +0000 UTC m=+1566.073302591"
Jul 23 11:52:34.437587 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:34.437552    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:52:37.785299 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:37.785267    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:52:42.831620 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.831593    2574 generic.go:356] "Generic (PLEG): container finished" podID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerID="dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb" exitCode=0
Jul 23 11:52:42.832052 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.831694    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerDied","Data":"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"}
Jul 23 11:52:42.832126 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.832111    2574 scope.go:117] "RemoveContainer" containerID="dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"
Jul 23 11:52:42.833107 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.833085    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:42.833152 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.833143    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:42.833185 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:42.833171    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 23 11:52:43.751807 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:43.751776    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:43.838606 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:43.838575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerStarted","Data":"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"}
Jul 23 11:52:44.008611 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:44.008516    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:44.009781 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:44.009755    2574 patch_prober.go:28] interesting pod/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 11:52:44.009877 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:44.009808    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 11:52:45.008810 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:45.008771    2574 patch_prober.go:28] interesting pod/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 11:52:45.009237 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:45.008829    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 11:52:46.008454 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:46.008414    2574 patch_prober.go:28] interesting pod/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 11:52:46.008678 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:46.008509    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 11:52:47.008102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:47.008060    2574 patch_prober.go:28] interesting pod/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 23 11:52:47.008537 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:47.008119    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 23 11:52:48.012109 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:48.012078    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:48.012512 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:48.012492    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:52:48.013146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:52:48.013129    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:53:05.245069 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.245029    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:53:05.245651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.245265    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" podUID="22787dd8-7b63-4f13-ae6b-e5b032256469" containerName="maas-api" containerID="cri-o://e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411" gracePeriod=30
Jul 23 11:53:05.502137 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.502072    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:53:05.588109 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.588075    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") pod \"22787dd8-7b63-4f13-ae6b-e5b032256469\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") "
Jul 23 11:53:05.588302 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.588197    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca\") pod \"22787dd8-7b63-4f13-ae6b-e5b032256469\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") "
Jul 23 11:53:05.588302 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.588221    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpfzs\" (UniqueName: \"kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs\") pod \"22787dd8-7b63-4f13-ae6b-e5b032256469\" (UID: \"22787dd8-7b63-4f13-ae6b-e5b032256469\") "
Jul 23 11:53:05.588574 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.588545    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22787dd8-7b63-4f13-ae6b-e5b032256469" (UID: "22787dd8-7b63-4f13-ae6b-e5b032256469"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:53:05.590637 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.590604    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs" (OuterVolumeSpecName: "kube-api-access-xpfzs") pod "22787dd8-7b63-4f13-ae6b-e5b032256469" (UID: "22787dd8-7b63-4f13-ae6b-e5b032256469"). InnerVolumeSpecName "kube-api-access-xpfzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:53:05.590637 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.590622    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22787dd8-7b63-4f13-ae6b-e5b032256469" (UID: "22787dd8-7b63-4f13-ae6b-e5b032256469"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:53:05.689025 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.688992    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpfzs\" (UniqueName: \"kubernetes.io/projected/22787dd8-7b63-4f13-ae6b-e5b032256469-kube-api-access-xpfzs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:05.689025 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.689028    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22787dd8-7b63-4f13-ae6b-e5b032256469-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:05.689239 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.689041    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22787dd8-7b63-4f13-ae6b-e5b032256469-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:05.945423 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.945391    2574 generic.go:356] "Generic (PLEG): container finished" podID="22787dd8-7b63-4f13-ae6b-e5b032256469" containerID="e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411" exitCode=0
Jul 23 11:53:05.945656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.945457    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"
Jul 23 11:53:05.945656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.945486    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" event={"ID":"22787dd8-7b63-4f13-ae6b-e5b032256469","Type":"ContainerDied","Data":"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"}
Jul 23 11:53:05.945656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.945542    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r" event={"ID":"22787dd8-7b63-4f13-ae6b-e5b032256469","Type":"ContainerDied","Data":"d5f38f6beb8dae1d7dd655601a89781dfd714e57d06337ea84263e85ee6b35c8"}
Jul 23 11:53:05.945656 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.945566    2574 scope.go:117] "RemoveContainer" containerID="e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"
Jul 23 11:53:05.955476 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.955449    2574 scope.go:117] "RemoveContainer" containerID="e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"
Jul 23 11:53:05.955743 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:05.955720    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411\": container with ID starting with e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411 not found: ID does not exist" containerID="e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"
Jul 23 11:53:05.955844 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.955754    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411"} err="failed to get container status \"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411\": rpc error: code = NotFound desc = could not find container \"e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411\": container with ID starting with e103f673596290431df83eca2589b70e0f0a47f44fa2322041f0c16e9b116411 not found: ID does not exist"
Jul 23 11:53:05.966896 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.966873    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:53:05.970721 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:05.970701    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2a4ab95e-54b68d99cf-tns6r"]
Jul 23 11:53:06.298407 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:06.298322    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22787dd8-7b63-4f13-ae6b-e5b032256469" path="/var/lib/kubelet/pods/22787dd8-7b63-4f13-ae6b-e5b032256469/volumes"
Jul 23 11:53:08.131501 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:08.131452    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:53:09.956199 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:09.956164    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:53:09.956631 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:09.956419    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy" containerID="cri-o://018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926" gracePeriod=30
Jul 23 11:53:15.211251 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.211228    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:53:15.379105 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379070    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379115    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379145    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76pqs\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379162    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379210    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379263    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379307 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379298    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379347    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379384    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket\") pod \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\" (UID: \"a82236d2-4405-4e7f-ad70-9b485b9bb8c6\") "
Jul 23 11:53:15.379651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379610    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:53:15.379863 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379819    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.380582 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.379951    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:53:15.380582 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.380262    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:53:15.380582 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.380450    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:53:15.380582 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.380517    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:53:15.381803 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.381777    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 11:53:15.381914 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.381865    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:53:15.381980 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.381961    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:53:15.382022 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.381979    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs" (OuterVolumeSpecName: "kube-api-access-76pqs") pod "a82236d2-4405-4e7f-ad70-9b485b9bb8c6" (UID: "a82236d2-4405-4e7f-ad70-9b485b9bb8c6"). InnerVolumeSpecName "kube-api-access-76pqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:53:15.480834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480799    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-data\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.480834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480828    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-credential-socket\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.480834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480837    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istiod-ca-cert\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.481047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480846    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-workload-socket\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.481047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480853    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-token\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.481047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480862    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-podinfo\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.481047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480871    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76pqs\" (UniqueName: \"kubernetes.io/projected/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-kube-api-access-76pqs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.481047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.480880    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a82236d2-4405-4e7f-ad70-9b485b9bb8c6-istio-envoy\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:15.995146 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.995113    2574 generic.go:356] "Generic (PLEG): container finished" podID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerID="018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926" exitCode=0
Jul 23 11:53:15.995325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.995163    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"
Jul 23 11:53:15.995325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.995162    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerDied","Data":"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"}
Jul 23 11:53:15.995325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.995293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb" event={"ID":"a82236d2-4405-4e7f-ad70-9b485b9bb8c6","Type":"ContainerDied","Data":"3838fa2742f20d6bb73968cd5faf1013a77fad257c6921008ca19660925469c0"}
Jul 23 11:53:15.995325 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:15.995324    2574 scope.go:117] "RemoveContainer" containerID="018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"
Jul 23 11:53:16.004838 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.004819    2574 scope.go:117] "RemoveContainer" containerID="dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"
Jul 23 11:53:16.012817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.012794    2574 scope.go:117] "RemoveContainer" containerID="018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"
Jul 23 11:53:16.013047 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:16.013024    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926\": container with ID starting with 018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926 not found: ID does not exist" containerID="018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"
Jul 23 11:53:16.013103 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.013061    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926"} err="failed to get container status \"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926\": rpc error: code = NotFound desc = could not find container \"018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926\": container with ID starting with 018234774d1c52bedc11ad22be541b5bd001c720d35ed1071b6c8c20e8396926 not found: ID does not exist"
Jul 23 11:53:16.013103 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.013082    2574 scope.go:117] "RemoveContainer" containerID="dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"
Jul 23 11:53:16.013309 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:16.013291    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb\": container with ID starting with dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb not found: ID does not exist" containerID="dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"
Jul 23 11:53:16.013364 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.013320    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb"} err="failed to get container status \"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb\": rpc error: code = NotFound desc = could not find container \"dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb\": container with ID starting with dc656718a93ae57bedc0f164ca3825a343a8f38ae2db7a1eb7382a99bd92a2eb not found: ID does not exist"
Jul 23 11:53:16.016959 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.016936    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:53:16.020058 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.020040    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2a4ab95e-openshift-default-85974fddbf-4prfb"]
Jul 23 11:53:16.224712 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.224679    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:16.225232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225216    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:16.225285 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225247    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:16.225285 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225275    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22787dd8-7b63-4f13-ae6b-e5b032256469" containerName="maas-api"
Jul 23 11:53:16.225285 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225283    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="22787dd8-7b63-4f13-ae6b-e5b032256469" containerName="maas-api"
Jul 23 11:53:16.225398 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225372    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="22787dd8-7b63-4f13-ae6b-e5b032256469" containerName="maas-api"
Jul 23 11:53:16.225398 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.225388    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:16.230215 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.230196    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.232728 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.232707    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d6a2356d-serving-cert\""
Jul 23 11:53:16.240534 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.240507    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:16.298214 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.298123    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" path="/var/lib/kubelet/pods/a82236d2-4405-4e7f-ad70-9b485b9bb8c6/volumes"
Jul 23 11:53:16.388634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.388600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj7vq\" (UniqueName: \"kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.388634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.388636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.388834 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.388727    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.489705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.489669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj7vq\" (UniqueName: \"kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.489705 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.489707    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.489906 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.489757    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.490355 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.490333    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.492284 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.492263    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.507389 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.507367    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj7vq\" (UniqueName: \"kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq\") pod \"maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.541871 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.541845    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:16.687107 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:16.687078    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:16.689895 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:53:16.689870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd308bc65_6444_4a43_b9ba_8917b5376f29.slice/crio-18596f6196844bad099e17bf088736a6ba0b85a8b21ef97683086d2537e95887 WatchSource:0}: Error finding container 18596f6196844bad099e17bf088736a6ba0b85a8b21ef97683086d2537e95887: Status 404 returned error can't find the container with id 18596f6196844bad099e17bf088736a6ba0b85a8b21ef97683086d2537e95887
Jul 23 11:53:17.002064 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:17.001982    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" event={"ID":"d308bc65-6444-4a43-b9ba-8917b5376f29","Type":"ContainerStarted","Data":"18596f6196844bad099e17bf088736a6ba0b85a8b21ef97683086d2537e95887"}
Jul 23 11:53:18.008697 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:18.008656    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" event={"ID":"d308bc65-6444-4a43-b9ba-8917b5376f29","Type":"ContainerStarted","Data":"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"}
Jul 23 11:53:18.009102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:18.008721    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:18.025457 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:18.025413    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" podStartSLOduration=1.5920935 podStartE2EDuration="2.025398189s" totalImagesPullingTime="433.304689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:53:18.02498474 +0000 UTC m=+1612.310450508" watchObservedRunningTime="2026-07-23 11:53:18.025398189 +0000 UTC m=+1612.310863955"
Jul 23 11:53:20.541737 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:20.541695    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:53:24.018710 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:24.018676    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:51.617804 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:51.617767    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:51.618383 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:51.617996    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" podUID="d308bc65-6444-4a43-b9ba-8917b5376f29" containerName="maas-api" containerID="cri-o://17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd" gracePeriod=30
Jul 23 11:53:51.873592 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:51.873537    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:52.008170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.008139    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj7vq\" (UniqueName: \"kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq\") pod \"d308bc65-6444-4a43-b9ba-8917b5376f29\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") "
Jul 23 11:53:52.008314 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.008249    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls\") pod \"d308bc65-6444-4a43-b9ba-8917b5376f29\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") "
Jul 23 11:53:52.008314 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.008287    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca\") pod \"d308bc65-6444-4a43-b9ba-8917b5376f29\" (UID: \"d308bc65-6444-4a43-b9ba-8917b5376f29\") "
Jul 23 11:53:52.008711 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.008689    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d308bc65-6444-4a43-b9ba-8917b5376f29" (UID: "d308bc65-6444-4a43-b9ba-8917b5376f29"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:53:52.010439 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.010410    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d308bc65-6444-4a43-b9ba-8917b5376f29" (UID: "d308bc65-6444-4a43-b9ba-8917b5376f29"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:53:52.010439 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.010429    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq" (OuterVolumeSpecName: "kube-api-access-fj7vq") pod "d308bc65-6444-4a43-b9ba-8917b5376f29" (UID: "d308bc65-6444-4a43-b9ba-8917b5376f29"). InnerVolumeSpecName "kube-api-access-fj7vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:53:52.109188 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.109155    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d308bc65-6444-4a43-b9ba-8917b5376f29-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:52.109188 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.109182    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d308bc65-6444-4a43-b9ba-8917b5376f29-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:52.109188 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.109192    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj7vq\" (UniqueName: \"kubernetes.io/projected/d308bc65-6444-4a43-b9ba-8917b5376f29-kube-api-access-fj7vq\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:53:52.186232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.186147    2574 generic.go:356] "Generic (PLEG): container finished" podID="d308bc65-6444-4a43-b9ba-8917b5376f29" containerID="17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd" exitCode=0
Jul 23 11:53:52.186232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.186191    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" event={"ID":"d308bc65-6444-4a43-b9ba-8917b5376f29","Type":"ContainerDied","Data":"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"}
Jul 23 11:53:52.186232 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.186210    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"
Jul 23 11:53:52.186456 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.186245    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz" event={"ID":"d308bc65-6444-4a43-b9ba-8917b5376f29","Type":"ContainerDied","Data":"18596f6196844bad099e17bf088736a6ba0b85a8b21ef97683086d2537e95887"}
Jul 23 11:53:52.186456 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.186276    2574 scope.go:117] "RemoveContainer" containerID="17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"
Jul 23 11:53:52.196049 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.196032    2574 scope.go:117] "RemoveContainer" containerID="17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"
Jul 23 11:53:52.196320 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:52.196301    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd\": container with ID starting with 17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd not found: ID does not exist" containerID="17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"
Jul 23 11:53:52.196381 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.196329    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd"} err="failed to get container status \"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd\": rpc error: code = NotFound desc = could not find container \"17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd\": container with ID starting with 17f0eb6bf4a2f688f442d704118b865ce7d8b9bc56824530d0e9fdc951ae83dd not found: ID does not exist"
Jul 23 11:53:52.207315 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.207290    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:52.210979 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.210960    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a2356d-55555d59db-6vrbz"]
Jul 23 11:53:52.298376 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:52.298346    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d308bc65-6444-4a43-b9ba-8917b5376f29" path="/var/lib/kubelet/pods/d308bc65-6444-4a43-b9ba-8917b5376f29/volumes"
Jul 23 11:53:54.424772 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:54.424735    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:53:59.431780 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.431741    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:53:59.432311 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432294    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:59.432390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432314    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:59.432390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432335    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d308bc65-6444-4a43-b9ba-8917b5376f29" containerName="maas-api"
Jul 23 11:53:59.432390 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432343    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d308bc65-6444-4a43-b9ba-8917b5376f29" containerName="maas-api"
Jul 23 11:53:59.432507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432440    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82236d2-4405-4e7f-ad70-9b485b9bb8c6" containerName="istio-proxy"
Jul 23 11:53:59.432507 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.432453    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d308bc65-6444-4a43-b9ba-8917b5376f29" containerName="maas-api"
Jul 23 11:53:59.435683 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.435652    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.438008 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.437988    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-aee44b3e-serving-cert\""
Jul 23 11:53:59.447514 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.447487    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:53:59.578932 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.578896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.579119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.578938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.579119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.579018    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnfdk\" (UniqueName: \"kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.679841 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.679803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnfdk\" (UniqueName: \"kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.680029 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.679896    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.680029 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.679920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.680160 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:59.680054    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-serving-cert: secret "maas-api-e2e-ait-aee44b3e-serving-cert" not found
Jul 23 11:53:59.680160 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:53:59.680144    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls podName:846c4402-ef3a-436c-a993-87dcdf1c90f6 nodeName:}" failed. No retries permitted until 2026-07-23 11:54:00.180121245 +0000 UTC m=+1654.465587197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls") pod "maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" (UID: "846c4402-ef3a-436c-a993-87dcdf1c90f6") : secret "maas-api-e2e-ait-aee44b3e-serving-cert" not found
Jul 23 11:53:59.680570 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.680554    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:53:59.688817 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:53:59.688759    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnfdk\" (UniqueName: \"kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:00.185810 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:00.185775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:00.188355 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:00.188328    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") pod \"maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:00.349713 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:00.349680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:00.483813 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:00.483787    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:54:00.485580 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:54:00.485553    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod846c4402_ef3a_436c_a993_87dcdf1c90f6.slice/crio-44297bf70fd43093adc7268f116f53b8a1d7137c9976940ae5ac5942ea436000 WatchSource:0}: Error finding container 44297bf70fd43093adc7268f116f53b8a1d7137c9976940ae5ac5942ea436000: Status 404 returned error can't find the container with id 44297bf70fd43093adc7268f116f53b8a1d7137c9976940ae5ac5942ea436000
Jul 23 11:54:01.228590 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:01.228498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" event={"ID":"846c4402-ef3a-436c-a993-87dcdf1c90f6","Type":"ContainerStarted","Data":"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"}
Jul 23 11:54:01.228742 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:01.228609    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" event={"ID":"846c4402-ef3a-436c-a993-87dcdf1c90f6","Type":"ContainerStarted","Data":"44297bf70fd43093adc7268f116f53b8a1d7137c9976940ae5ac5942ea436000"}
Jul 23 11:54:01.228742 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:01.228639    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:01.249571 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:01.249522    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" podStartSLOduration=1.88295977 podStartE2EDuration="2.249507862s" totalImagesPullingTime="366.548092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:54:01.246676222 +0000 UTC m=+1655.532141988" watchObservedRunningTime="2026-07-23 11:54:01.249507862 +0000 UTC m=+1655.534973628"
Jul 23 11:54:02.931815 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:02.931780    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:54:07.239755 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:07.239725    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:35.627951 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.627912    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:54:35.628405 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.628185    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" podUID="846c4402-ef3a-436c-a993-87dcdf1c90f6" containerName="maas-api" containerID="cri-o://0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8" gracePeriod=30
Jul 23 11:54:35.886392 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.886339    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:35.998323 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.998288    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnfdk\" (UniqueName: \"kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk\") pod \"846c4402-ef3a-436c-a993-87dcdf1c90f6\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") "
Jul 23 11:54:35.998557 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.998387    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") pod \"846c4402-ef3a-436c-a993-87dcdf1c90f6\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") "
Jul 23 11:54:35.998557 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.998406    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca\") pod \"846c4402-ef3a-436c-a993-87dcdf1c90f6\" (UID: \"846c4402-ef3a-436c-a993-87dcdf1c90f6\") "
Jul 23 11:54:35.998831 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:35.998797    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "846c4402-ef3a-436c-a993-87dcdf1c90f6" (UID: "846c4402-ef3a-436c-a993-87dcdf1c90f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:54:36.000694 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.000671    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "846c4402-ef3a-436c-a993-87dcdf1c90f6" (UID: "846c4402-ef3a-436c-a993-87dcdf1c90f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:54:36.000803 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.000716    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk" (OuterVolumeSpecName: "kube-api-access-rnfdk") pod "846c4402-ef3a-436c-a993-87dcdf1c90f6" (UID: "846c4402-ef3a-436c-a993-87dcdf1c90f6"). InnerVolumeSpecName "kube-api-access-rnfdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:54:36.099767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.099725    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/846c4402-ef3a-436c-a993-87dcdf1c90f6-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:54:36.099767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.099755    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/846c4402-ef3a-436c-a993-87dcdf1c90f6-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:54:36.099767 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.099765    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnfdk\" (UniqueName: \"kubernetes.io/projected/846c4402-ef3a-436c-a993-87dcdf1c90f6-kube-api-access-rnfdk\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:54:36.394605 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.394566    2574 generic.go:356] "Generic (PLEG): container finished" podID="846c4402-ef3a-436c-a993-87dcdf1c90f6" containerID="0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8" exitCode=0
Jul 23 11:54:36.394762 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.394612    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"
Jul 23 11:54:36.394762 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.394637    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" event={"ID":"846c4402-ef3a-436c-a993-87dcdf1c90f6","Type":"ContainerDied","Data":"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"}
Jul 23 11:54:36.394762 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.394690    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98" event={"ID":"846c4402-ef3a-436c-a993-87dcdf1c90f6","Type":"ContainerDied","Data":"44297bf70fd43093adc7268f116f53b8a1d7137c9976940ae5ac5942ea436000"}
Jul 23 11:54:36.394762 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.394720    2574 scope.go:117] "RemoveContainer" containerID="0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"
Jul 23 11:54:36.403396 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.403366    2574 scope.go:117] "RemoveContainer" containerID="0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"
Jul 23 11:54:36.403697 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:54:36.403663    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8\": container with ID starting with 0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8 not found: ID does not exist" containerID="0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"
Jul 23 11:54:36.403745 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.403708    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8"} err="failed to get container status \"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8\": rpc error: code = NotFound desc = could not find container \"0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8\": container with ID starting with 0415676c179d24044c3a90a0c1885970481b10bd83110eed7c11cb8c3cedcab8 not found: ID does not exist"
Jul 23 11:54:36.411821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.411798    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:54:36.415375 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:36.415348    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-aee44b3e-79b45c5b8c-8bn98"]
Jul 23 11:54:38.230272 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:38.230237    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:54:38.297162 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:38.297120    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="846c4402-ef3a-436c-a993-87dcdf1c90f6" path="/var/lib/kubelet/pods/846c4402-ef3a-436c-a993-87dcdf1c90f6/volumes"
Jul 23 11:54:44.688828 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.688796    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:54:44.689306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.689159    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="846c4402-ef3a-436c-a993-87dcdf1c90f6" containerName="maas-api"
Jul 23 11:54:44.689306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.689170    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="846c4402-ef3a-436c-a993-87dcdf1c90f6" containerName="maas-api"
Jul 23 11:54:44.689306 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.689229    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="846c4402-ef3a-436c-a993-87dcdf1c90f6" containerName="maas-api"
Jul 23 11:54:44.693419 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.693397    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.695850 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.695829    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a4beeb76-serving-cert\""
Jul 23 11:54:44.703653 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.703612    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:54:44.775716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.775681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm2gs\" (UniqueName: \"kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.775716 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.775722    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.775976 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.775773    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.876305 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.876271    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2gs\" (UniqueName: \"kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.876305 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.876313    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.876594 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:54:44.876417    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-serving-cert: secret "maas-api-e2e-derive-a4beeb76-serving-cert" not found
Jul 23 11:54:44.876594 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.876435    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.876594 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:54:44.876510    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls podName:f536ae6a-eaa7-464f-a634-95501c1ff9c1 nodeName:}" failed. No retries permitted until 2026-07-23 11:54:45.376458753 +0000 UTC m=+1699.661924497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls") pod "maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" (UID: "f536ae6a-eaa7-464f-a634-95501c1ff9c1") : secret "maas-api-e2e-derive-a4beeb76-serving-cert" not found
Jul 23 11:54:44.877113 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.877091    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:44.890134 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:44.890111    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2gs\" (UniqueName: \"kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:45.380336 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:45.380304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:45.382960 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:45.382937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") pod \"maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:45.605601 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:45.605558    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:45.739035 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:45.739009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:54:45.741156 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:54:45.741123    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf536ae6a_eaa7_464f_a634_95501c1ff9c1.slice/crio-451755d4995a578a7c0e2b3c10d657927304f64bb71ed0760fd8fa17a052d6ad WatchSource:0}: Error finding container 451755d4995a578a7c0e2b3c10d657927304f64bb71ed0760fd8fa17a052d6ad: Status 404 returned error can't find the container with id 451755d4995a578a7c0e2b3c10d657927304f64bb71ed0760fd8fa17a052d6ad
Jul 23 11:54:46.443690 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:46.443647    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" event={"ID":"f536ae6a-eaa7-464f-a634-95501c1ff9c1","Type":"ContainerStarted","Data":"451755d4995a578a7c0e2b3c10d657927304f64bb71ed0760fd8fa17a052d6ad"}
Jul 23 11:54:47.450184 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:47.450103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" event={"ID":"f536ae6a-eaa7-464f-a634-95501c1ff9c1","Type":"ContainerStarted","Data":"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"}
Jul 23 11:54:47.450559 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:47.450234    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:54:47.467570 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:47.467523    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" podStartSLOduration=2.57914802 podStartE2EDuration="3.467509462s" totalImagesPullingTime="888.361442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:54:47.465709443 +0000 UTC m=+1701.751175209" watchObservedRunningTime="2026-07-23 11:54:47.467509462 +0000 UTC m=+1701.752975228"
Jul 23 11:54:50.842828 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:50.842789    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:54:53.461021 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:54:53.460989    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:55:20.005607 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.005568    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:55:20.006255 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.005833    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" podUID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" containerName="maas-api" containerID="cri-o://fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817" gracePeriod=30
Jul 23 11:55:20.262631 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.262568    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:55:20.299859 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.299834    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") pod \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") "
Jul 23 11:55:20.300035 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.299903    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm2gs\" (UniqueName: \"kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs\") pod \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") "
Jul 23 11:55:20.300035 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.299936    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca\") pod \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\" (UID: \"f536ae6a-eaa7-464f-a634-95501c1ff9c1\") "
Jul 23 11:55:20.300323 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.300293    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f536ae6a-eaa7-464f-a634-95501c1ff9c1" (UID: "f536ae6a-eaa7-464f-a634-95501c1ff9c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:55:20.302085 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.302059    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f536ae6a-eaa7-464f-a634-95501c1ff9c1" (UID: "f536ae6a-eaa7-464f-a634-95501c1ff9c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:55:20.302312 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.302292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs" (OuterVolumeSpecName: "kube-api-access-zm2gs") pod "f536ae6a-eaa7-464f-a634-95501c1ff9c1" (UID: "f536ae6a-eaa7-464f-a634-95501c1ff9c1"). InnerVolumeSpecName "kube-api-access-zm2gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:55:20.400744 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.400703    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm2gs\" (UniqueName: \"kubernetes.io/projected/f536ae6a-eaa7-464f-a634-95501c1ff9c1-kube-api-access-zm2gs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:55:20.400744 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.400731    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f536ae6a-eaa7-464f-a634-95501c1ff9c1-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:55:20.400744 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.400741    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f536ae6a-eaa7-464f-a634-95501c1ff9c1-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:55:20.604575 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.604545    2574 generic.go:356] "Generic (PLEG): container finished" podID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" containerID="fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817" exitCode=0
Jul 23 11:55:20.604750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.604588    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"
Jul 23 11:55:20.604750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.604613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" event={"ID":"f536ae6a-eaa7-464f-a634-95501c1ff9c1","Type":"ContainerDied","Data":"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"}
Jul 23 11:55:20.604750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.604654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg" event={"ID":"f536ae6a-eaa7-464f-a634-95501c1ff9c1","Type":"ContainerDied","Data":"451755d4995a578a7c0e2b3c10d657927304f64bb71ed0760fd8fa17a052d6ad"}
Jul 23 11:55:20.604750 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.604677    2574 scope.go:117] "RemoveContainer" containerID="fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"
Jul 23 11:55:20.614353 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.614337    2574 scope.go:117] "RemoveContainer" containerID="fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"
Jul 23 11:55:20.614613 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:55:20.614592    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817\": container with ID starting with fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817 not found: ID does not exist" containerID="fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"
Jul 23 11:55:20.614680 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.614619    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817"} err="failed to get container status \"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817\": rpc error: code = NotFound desc = could not find container \"fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817\": container with ID starting with fe3c56a116e459a1def37b4378dda8f321552ba53e561f4864cd58a1a9ba2817 not found: ID does not exist"
Jul 23 11:55:20.628022 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.628000    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:55:20.634070 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:20.634048    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a4beeb76-5f4fc568b5-4fxpg"]
Jul 23 11:55:22.298007 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:22.297975    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" path="/var/lib/kubelet/pods/f536ae6a-eaa7-464f-a634-95501c1ff9c1/volumes"
Jul 23 11:55:22.628287 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:55:22.628254    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:56:26.302910 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:56:26.302881    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:56:26.310119 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:56:26.310096    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:56:26.316708 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:56:26.316690    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 11:56:26.322949 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:56:26.322929    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 11:56:52.634115 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:56:52.634079    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:57:04.352429 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:04.352394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:57:08.634404 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.634363    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:08.634924 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.634791    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" containerName="maas-api"
Jul 23 11:57:08.634924 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.634806    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" containerName="maas-api"
Jul 23 11:57:08.634924 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.634876    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f536ae6a-eaa7-464f-a634-95501c1ff9c1" containerName="maas-api"
Jul 23 11:57:08.638441 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.638417    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.641318 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.641298    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bd7354bb-serving-cert\""
Jul 23 11:57:08.658374 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.658346    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:08.768191 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.768152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.768386 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.768233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.768386 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.768252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkjl\" (UniqueName: \"kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.869197 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.869161    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.869197 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.869198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkjl\" (UniqueName: \"kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.869428 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.869251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.869935 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.869907    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.871996 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.871970    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.884293 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.884266    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkjl\" (UniqueName: \"kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl\") pod \"maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:08.950624 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:08.950528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:09.106561 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:09.106485    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:09.108799 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:57:09.108761    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393ed29b_312c_4032_9fbc_ae94de6d2a08.slice/crio-1660889facb727dde6397a5507c0c5a7e1a347f0e939623a078734fd1d1221f6 WatchSource:0}: Error finding container 1660889facb727dde6397a5507c0c5a7e1a347f0e939623a078734fd1d1221f6: Status 404 returned error can't find the container with id 1660889facb727dde6397a5507c0c5a7e1a347f0e939623a078734fd1d1221f6
Jul 23 11:57:10.116281 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:10.116243    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" event={"ID":"393ed29b-312c-4032-9fbc-ae94de6d2a08","Type":"ContainerStarted","Data":"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"}
Jul 23 11:57:10.116753 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:10.116319    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" event={"ID":"393ed29b-312c-4032-9fbc-ae94de6d2a08","Type":"ContainerStarted","Data":"1660889facb727dde6397a5507c0c5a7e1a347f0e939623a078734fd1d1221f6"}
Jul 23 11:57:10.116753 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:10.116361    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:10.138352 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:10.138302    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" podStartSLOduration=1.5678282970000001 podStartE2EDuration="2.138287078s" totalImagesPullingTime="570.458781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:57:10.136916104 +0000 UTC m=+1844.422381869" watchObservedRunningTime="2026-07-23 11:57:10.138287078 +0000 UTC m=+1844.423752843"
Jul 23 11:57:16.127105 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:16.127073    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:18.335613 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:18.335574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:57:30.588387 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.588353    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:30.589028 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.588609    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" podUID="393ed29b-312c-4032-9fbc-ae94de6d2a08" containerName="maas-api" containerID="cri-o://acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db" gracePeriod=30
Jul 23 11:57:30.842629 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.842571    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:30.873283 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.873255    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca\") pod \"393ed29b-312c-4032-9fbc-ae94de6d2a08\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") "
Jul 23 11:57:30.873492 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.873305    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls\") pod \"393ed29b-312c-4032-9fbc-ae94de6d2a08\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") "
Jul 23 11:57:30.873492 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.873327    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnkjl\" (UniqueName: \"kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl\") pod \"393ed29b-312c-4032-9fbc-ae94de6d2a08\" (UID: \"393ed29b-312c-4032-9fbc-ae94de6d2a08\") "
Jul 23 11:57:30.873706 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.873666    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "393ed29b-312c-4032-9fbc-ae94de6d2a08" (UID: "393ed29b-312c-4032-9fbc-ae94de6d2a08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:57:30.875667 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.875631    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "393ed29b-312c-4032-9fbc-ae94de6d2a08" (UID: "393ed29b-312c-4032-9fbc-ae94de6d2a08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:57:30.875772 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.875645    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl" (OuterVolumeSpecName: "kube-api-access-tnkjl") pod "393ed29b-312c-4032-9fbc-ae94de6d2a08" (UID: "393ed29b-312c-4032-9fbc-ae94de6d2a08"). InnerVolumeSpecName "kube-api-access-tnkjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:57:30.974721 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.974689    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/393ed29b-312c-4032-9fbc-ae94de6d2a08-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:57:30.974721 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.974715    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/393ed29b-312c-4032-9fbc-ae94de6d2a08-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:57:30.974721 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:30.974724    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnkjl\" (UniqueName: \"kubernetes.io/projected/393ed29b-312c-4032-9fbc-ae94de6d2a08-kube-api-access-tnkjl\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:57:31.214425 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.214326    2574 generic.go:356] "Generic (PLEG): container finished" podID="393ed29b-312c-4032-9fbc-ae94de6d2a08" containerID="acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db" exitCode=0
Jul 23 11:57:31.214425 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.214381    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"
Jul 23 11:57:31.214652 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.214429    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" event={"ID":"393ed29b-312c-4032-9fbc-ae94de6d2a08","Type":"ContainerDied","Data":"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"}
Jul 23 11:57:31.214652 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.214512    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5" event={"ID":"393ed29b-312c-4032-9fbc-ae94de6d2a08","Type":"ContainerDied","Data":"1660889facb727dde6397a5507c0c5a7e1a347f0e939623a078734fd1d1221f6"}
Jul 23 11:57:31.214652 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.214537    2574 scope.go:117] "RemoveContainer" containerID="acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"
Jul 23 11:57:31.223981 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.223960    2574 scope.go:117] "RemoveContainer" containerID="acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"
Jul 23 11:57:31.224246 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:57:31.224228    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db\": container with ID starting with acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db not found: ID does not exist" containerID="acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"
Jul 23 11:57:31.224289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.224255    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db"} err="failed to get container status \"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db\": rpc error: code = NotFound desc = could not find container \"acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db\": container with ID starting with acfef95e4967e6edc6f6ba04aac009f384b3adc036156f5e3476ffe04cf6a0db not found: ID does not exist"
Jul 23 11:57:31.236854 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.236829    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:31.238496 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:31.238455    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bd7354bb-64c44ff74d-zk5n5"]
Jul 23 11:57:32.298481 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:32.298422    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="393ed29b-312c-4032-9fbc-ae94de6d2a08" path="/var/lib/kubelet/pods/393ed29b-312c-4032-9fbc-ae94de6d2a08/volumes"
Jul 23 11:57:33.437696 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:57:33.437653    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:58:04.536646 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.536615    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:58:04.539158 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.537119    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="393ed29b-312c-4032-9fbc-ae94de6d2a08" containerName="maas-api"
Jul 23 11:58:04.539158 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.537133    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="393ed29b-312c-4032-9fbc-ae94de6d2a08" containerName="maas-api"
Jul 23 11:58:04.539158 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.537194    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="393ed29b-312c-4032-9fbc-ae94de6d2a08" containerName="maas-api"
Jul 23 11:58:04.539992 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.539974    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.542820 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.542801    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b7dc3a03-serving-cert\""
Jul 23 11:58:04.554692 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.554666    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:58:04.675289 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.675245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.675544 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.675345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.675544 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.675371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhzmp\" (UniqueName: \"kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.776448 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.776410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.776695 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.776518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.776695 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.776543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhzmp\" (UniqueName: \"kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.777170 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.777140    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.779097 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.779076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.784508 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.784487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhzmp\" (UniqueName: \"kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp\") pod \"maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:04.850718 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:04.850694    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:05.200494 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:05.200417    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:58:05.202895 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:58:05.202867    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3d1cec_19eb_49c1_902a_d600911f9405.slice/crio-8e7308103e26184557a384d6488872b9ce1eb6b19d3aa606f032b3a31ecfecc4 WatchSource:0}: Error finding container 8e7308103e26184557a384d6488872b9ce1eb6b19d3aa606f032b3a31ecfecc4: Status 404 returned error can't find the container with id 8e7308103e26184557a384d6488872b9ce1eb6b19d3aa606f032b3a31ecfecc4
Jul 23 11:58:05.204214 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:05.204195    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 11:58:05.375240 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:05.375200    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" event={"ID":"6b3d1cec-19eb-49c1-902a-d600911f9405","Type":"ContainerStarted","Data":"8e7308103e26184557a384d6488872b9ce1eb6b19d3aa606f032b3a31ecfecc4"}
Jul 23 11:58:06.381299 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:06.381268    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" event={"ID":"6b3d1cec-19eb-49c1-902a-d600911f9405","Type":"ContainerStarted","Data":"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"}
Jul 23 11:58:06.381707 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:06.381416    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:06.398245 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:06.398194    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" podStartSLOduration=2.024706925 podStartE2EDuration="2.398176585s" totalImagesPullingTime="373.46966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:58:06.39794062 +0000 UTC m=+1900.683406387" watchObservedRunningTime="2026-07-23 11:58:06.398176585 +0000 UTC m=+1900.683642353"
Jul 23 11:58:08.838169 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:08.838136    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:58:12.391737 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:12.391708    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:58:23.428615 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.428577    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:58:23.433386 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.433359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.436944 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.436914    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b7dc3a03\"/\"openshift-service-ca.crt\""
Jul 23 11:58:23.436944 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.436924    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b7dc3a03\"/\"default-dockercfg-5n78w\""
Jul 23 11:58:23.437141 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.436983    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b7dc3a03\"/\"e2e-lifecycle-model-b7dc3a03-kserve-self-signed-certs\""
Jul 23 11:58:23.437141 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.436992    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b7dc3a03\"/\"kube-root-ca.crt\""
Jul 23 11:58:23.442504 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.442481    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:58:23.547256 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547225    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.547256 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.547505 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.547505 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.547505 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.547505 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.547435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5c4\" (UniqueName: \"kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.648887 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.648841    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.648887 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.648898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5c4\" (UniqueName: \"kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649133 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.648932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649133 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.648951    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649133 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.648996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649133 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.649017    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649374 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.649321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649434 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.649376    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.649546 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.649526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.651345 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.651324    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.651758 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.651742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.657587 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.657566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5c4\" (UniqueName: \"kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4\") pod \"e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") " pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.745846 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.745767    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:23.884003 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:23.883974    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:58:23.884512 ip-10-0-140-183 kubenswrapper[2574]: W0723 11:58:23.884481    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60844516_cd10_4b95_b7b4_fc0ab7f8d6c9.slice/crio-faa92e06337e6e09d52b0ba2444682d122d7ba47065d41bc975a59978f1c4bee WatchSource:0}: Error finding container faa92e06337e6e09d52b0ba2444682d122d7ba47065d41bc975a59978f1c4bee: Status 404 returned error can't find the container with id faa92e06337e6e09d52b0ba2444682d122d7ba47065d41bc975a59978f1c4bee
Jul 23 11:58:24.465879 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:24.465829    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" event={"ID":"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9","Type":"ContainerStarted","Data":"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"}
Jul 23 11:58:24.466276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:24.465886    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" event={"ID":"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9","Type":"ContainerStarted","Data":"faa92e06337e6e09d52b0ba2444682d122d7ba47065d41bc975a59978f1c4bee"}
Jul 23 11:58:24.484989 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:24.484942    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" podStartSLOduration=1.48492538 podStartE2EDuration="1.48492538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 11:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 11:58:24.483619389 +0000 UTC m=+1918.769085155" watchObservedRunningTime="2026-07-23 11:58:24.48492538 +0000 UTC m=+1918.770391147"
Jul 23 11:58:26.727416 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:26.727332    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:58:33.746043 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:33.746009    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:33.746503 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:33.746057    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:33.758506 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:33.758480    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:34.525271 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:34.525240    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:58:46.838722 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:58:46.838686    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:59:06.434623 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:06.434585    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:59:08.139738 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.139702    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:59:08.140115 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.139954    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" podUID="6b3d1cec-19eb-49c1-902a-d600911f9405" containerName="maas-api" containerID="cri-o://f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06" gracePeriod=30
Jul 23 11:59:08.384858 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.384836    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:59:08.459180 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.459091    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls\") pod \"6b3d1cec-19eb-49c1-902a-d600911f9405\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") "
Jul 23 11:59:08.459180 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.459166    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca\") pod \"6b3d1cec-19eb-49c1-902a-d600911f9405\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") "
Jul 23 11:59:08.459412 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.459205    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhzmp\" (UniqueName: \"kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp\") pod \"6b3d1cec-19eb-49c1-902a-d600911f9405\" (UID: \"6b3d1cec-19eb-49c1-902a-d600911f9405\") "
Jul 23 11:59:08.459605 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.459577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b3d1cec-19eb-49c1-902a-d600911f9405" (UID: "6b3d1cec-19eb-49c1-902a-d600911f9405"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 11:59:08.461341 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.461320    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b3d1cec-19eb-49c1-902a-d600911f9405" (UID: "6b3d1cec-19eb-49c1-902a-d600911f9405"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:59:08.461438 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.461422    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp" (OuterVolumeSpecName: "kube-api-access-fhzmp") pod "6b3d1cec-19eb-49c1-902a-d600911f9405" (UID: "6b3d1cec-19eb-49c1-902a-d600911f9405"). InnerVolumeSpecName "kube-api-access-fhzmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:59:08.560102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.560062    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b3d1cec-19eb-49c1-902a-d600911f9405-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:08.560102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.560096    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhzmp\" (UniqueName: \"kubernetes.io/projected/6b3d1cec-19eb-49c1-902a-d600911f9405-kube-api-access-fhzmp\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:08.560102 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.560107    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b3d1cec-19eb-49c1-902a-d600911f9405-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:08.671114 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.671075    2574 generic.go:356] "Generic (PLEG): container finished" podID="6b3d1cec-19eb-49c1-902a-d600911f9405" containerID="f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06" exitCode=0
Jul 23 11:59:08.671270 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.671123    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"
Jul 23 11:59:08.671270 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.671159    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" event={"ID":"6b3d1cec-19eb-49c1-902a-d600911f9405","Type":"ContainerDied","Data":"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"}
Jul 23 11:59:08.671270 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.671214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt" event={"ID":"6b3d1cec-19eb-49c1-902a-d600911f9405","Type":"ContainerDied","Data":"8e7308103e26184557a384d6488872b9ce1eb6b19d3aa606f032b3a31ecfecc4"}
Jul 23 11:59:08.671270 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.671236    2574 scope.go:117] "RemoveContainer" containerID="f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"
Jul 23 11:59:08.680334 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.680318    2574 scope.go:117] "RemoveContainer" containerID="f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"
Jul 23 11:59:08.680604 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:59:08.680588    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06\": container with ID starting with f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06 not found: ID does not exist" containerID="f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"
Jul 23 11:59:08.680651 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.680612    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06"} err="failed to get container status \"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06\": rpc error: code = NotFound desc = could not find container \"f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06\": container with ID starting with f11699ba43539dcf5be68bfae5fd685e463ef2f777fd869925491fe39da26d06 not found: ID does not exist"
Jul 23 11:59:08.691280 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.691257    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:59:08.697073 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:08.697049    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b7dc3a03-67ffc7c466-6fmkt"]
Jul 23 11:59:10.298031 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:10.297999    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3d1cec-19eb-49c1-902a-d600911f9405" path="/var/lib/kubelet/pods/6b3d1cec-19eb-49c1-902a-d600911f9405/volumes"
Jul 23 11:59:16.870657 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:16.870614    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 11:59:19.929623 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:19.929589    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:59:34.950519 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:34.950429    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" podUID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" containerName="main" containerID="cri-o://5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1" gracePeriod=45
Jul 23 11:59:35.212141 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.212069    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:59:35.300634 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300603    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj5c4\" (UniqueName: \"kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.300821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300671    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.300821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300757    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.300821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300784    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.300821 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300816    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.301047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.300843    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir\") pod \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\" (UID: \"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9\") "
Jul 23 11:59:35.301047 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.301015    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:59:35.301147 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.301074    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home" (OuterVolumeSpecName: "home") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:59:35.301202 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.301142    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:59:35.301202 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.301190    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.301283 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.301210    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.303167 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.303136    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm" (OuterVolumeSpecName: "dshm") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 11:59:35.303276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.303185    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 11:59:35.303276 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.303196    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4" (OuterVolumeSpecName: "kube-api-access-hj5c4") pod "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" (UID: "60844516-cd10-4b95-b7b4-fc0ab7f8d6c9"). InnerVolumeSpecName "kube-api-access-hj5c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 11:59:35.402135 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.402083    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj5c4\" (UniqueName: \"kubernetes.io/projected/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-kube-api-access-hj5c4\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.402135 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.402128    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.402135 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.402138    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.402135 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.402146    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 11:59:35.804993 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.804959    2574 generic.go:356] "Generic (PLEG): container finished" podID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" containerID="5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1" exitCode=0
Jul 23 11:59:35.805183 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.805035    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"
Jul 23 11:59:35.805183 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.805052    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" event={"ID":"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9","Type":"ContainerDied","Data":"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"}
Jul 23 11:59:35.805183 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.805099    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc" event={"ID":"60844516-cd10-4b95-b7b4-fc0ab7f8d6c9","Type":"ContainerDied","Data":"faa92e06337e6e09d52b0ba2444682d122d7ba47065d41bc975a59978f1c4bee"}
Jul 23 11:59:35.805183 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.805118    2574 scope.go:117] "RemoveContainer" containerID="5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"
Jul 23 11:59:35.814195 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.814178    2574 scope.go:117] "RemoveContainer" containerID="5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"
Jul 23 11:59:35.814484 ip-10-0-140-183 kubenswrapper[2574]: E0723 11:59:35.814438    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1\": container with ID starting with 5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1 not found: ID does not exist" containerID="5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"
Jul 23 11:59:35.814484 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.814485    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1"} err="failed to get container status \"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1\": rpc error: code = NotFound desc = could not find container \"5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1\": container with ID starting with 5508b15053782024f7c9038300cef19d8da565269ebced90f8eefa304e082ec1 not found: ID does not exist"
Jul 23 11:59:35.826659 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.826635    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:59:35.831035 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:35.831013    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b7dc3a03/e2e-lifecycle-model-b7dc3a03-kserve-7d65bc7f5f-pc6sc"]
Jul 23 11:59:36.298072 ip-10-0-140-183 kubenswrapper[2574]: I0723 11:59:36.298038    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" path="/var/lib/kubelet/pods/60844516-cd10-4b95-b7b4-fc0ab7f8d6c9/volumes"
Jul 23 12:00:12.140199 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:00:12.140163    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:00:25.259756 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:00:25.259716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:01:06.828760 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.828729    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829115    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" containerName="main"
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829133    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" containerName="main"
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829148    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3d1cec-19eb-49c1-902a-d600911f9405" containerName="maas-api"
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829153    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3d1cec-19eb-49c1-902a-d600911f9405" containerName="maas-api"
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829218    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3d1cec-19eb-49c1-902a-d600911f9405" containerName="maas-api"
Jul 23 12:01:06.829353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.829229    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="60844516-cd10-4b95-b7b4-fc0ab7f8d6c9" containerName="main"
Jul 23 12:01:06.832322 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.832303    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:06.836027 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.836002    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-f8adc6-serving-cert\""
Jul 23 12:01:06.844433 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.844394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:01:06.920529 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.920451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:06.920723 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.920543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:06.920723 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:06.920692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.021964 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.021927    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.022168 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.021986    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.022168 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.022008    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.022168 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:01:07.022103    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-serving-cert: secret "maas-api-e2e-inf-a-f8adc6-serving-cert" not found
Jul 23 12:01:07.022168 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:01:07.022156    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls podName:486b5e6f-2e00-4422-b1c8-c69f1325eaa1 nodeName:}" failed. No retries permitted until 2026-07-23 12:01:07.522140934 +0000 UTC m=+2081.807606678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls") pod "maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" (UID: "486b5e6f-2e00-4422-b1c8-c69f1325eaa1") : secret "maas-api-e2e-inf-a-f8adc6-serving-cert" not found
Jul 23 12:01:07.022772 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.022750    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.031247 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.031210    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.526910 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.526868    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.529417 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.529395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") pod \"maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.744028 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.743993    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:07.879749 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:07.879722    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:01:07.881454 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:01:07.881426    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486b5e6f_2e00_4422_b1c8_c69f1325eaa1.slice/crio-6f73e44281d8697df52360f33b87a0cb850968f92feb80fc5e7643495f07e1aa WatchSource:0}: Error finding container 6f73e44281d8697df52360f33b87a0cb850968f92feb80fc5e7643495f07e1aa: Status 404 returned error can't find the container with id 6f73e44281d8697df52360f33b87a0cb850968f92feb80fc5e7643495f07e1aa
Jul 23 12:01:08.236748 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:08.236703    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" event={"ID":"486b5e6f-2e00-4422-b1c8-c69f1325eaa1","Type":"ContainerStarted","Data":"6f73e44281d8697df52360f33b87a0cb850968f92feb80fc5e7643495f07e1aa"}
Jul 23 12:01:09.243135 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:09.243042    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" event={"ID":"486b5e6f-2e00-4422-b1c8-c69f1325eaa1","Type":"ContainerStarted","Data":"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"}
Jul 23 12:01:09.243520 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:09.243163    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:09.259751 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:09.259697    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" podStartSLOduration=2.221976881 podStartE2EDuration="3.25967848s" totalImagesPullingTime="1.037701599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:01:09.258815325 +0000 UTC m=+2083.544281090" watchObservedRunningTime="2026-07-23 12:01:09.25967848 +0000 UTC m=+2083.545144247"
Jul 23 12:01:14.233055 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.233020    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:01:14.821006 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.820961    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:01:14.825393 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.825369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:14.828067 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.828042    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-591d5c-serving-cert\""
Jul 23 12:01:14.834930 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.834906    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:01:14.900243 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.900202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:14.900458 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.900259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:14.900458 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:14.900316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5r6c\" (UniqueName: \"kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.001233 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.001200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.001403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.001270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.001403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.001296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5r6c\" (UniqueName: \"kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.001520 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:01:15.001429    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-serving-cert: secret "maas-api-e2e-inf-b-591d5c-serving-cert" not found
Jul 23 12:01:15.001563 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:01:15.001530    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls podName:9332080e-cce8-4e4b-aa5a-d545715dc7d7 nodeName:}" failed. No retries permitted until 2026-07-23 12:01:15.501507226 +0000 UTC m=+2089.786972976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls") pod "maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" (UID: "9332080e-cce8-4e4b-aa5a-d545715dc7d7") : secret "maas-api-e2e-inf-b-591d5c-serving-cert" not found
Jul 23 12:01:15.001952 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.001932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.011396 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.011373    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5r6c\" (UniqueName: \"kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.253284 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.253255    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:01:15.507159 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.507057    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.509700 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.509664    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") pod \"maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.739251 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.739216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:15.885004 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:15.884970    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:01:15.886606 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:01:15.886580    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9332080e_cce8_4e4b_aa5a_d545715dc7d7.slice/crio-a40ba453ba62114b629923ee5fd4db90a7f1575c7961584d600a8126e3b1dffd WatchSource:0}: Error finding container a40ba453ba62114b629923ee5fd4db90a7f1575c7961584d600a8126e3b1dffd: Status 404 returned error can't find the container with id a40ba453ba62114b629923ee5fd4db90a7f1575c7961584d600a8126e3b1dffd
Jul 23 12:01:16.276721 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:16.276682    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" event={"ID":"9332080e-cce8-4e4b-aa5a-d545715dc7d7","Type":"ContainerStarted","Data":"a40ba453ba62114b629923ee5fd4db90a7f1575c7961584d600a8126e3b1dffd"}
Jul 23 12:01:17.282648 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:17.282608    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" event={"ID":"9332080e-cce8-4e4b-aa5a-d545715dc7d7","Type":"ContainerStarted","Data":"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"}
Jul 23 12:01:17.283053 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:17.282676    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:17.299431 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:17.299384    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" podStartSLOduration=2.625040173 podStartE2EDuration="3.299367759s" totalImagesPullingTime="674.327586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:01:17.29779805 +0000 UTC m=+2091.583263818" watchObservedRunningTime="2026-07-23 12:01:17.299367759 +0000 UTC m=+2091.584833525"
Jul 23 12:01:23.293252 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:23.293214    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:01:26.331524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:26.331495    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:01:26.338441 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:26.338418    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:01:26.346208 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:26.346185    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:01:26.352724 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:26.352704    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:01:29.331413 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:29.331377    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:01:41.738314 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:41.738276    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:01:48.632060 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.632028    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:01:48.635674 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.635657    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.646990 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.646968    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-591d5c\"/\"openshift-service-ca.crt\""
Jul 23 12:01:48.647124 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.647105    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-591d5c\"/\"test-model-591d5c-kserve-self-signed-certs\""
Jul 23 12:01:48.652675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.652649    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-591d5c\"/\"kube-root-ca.crt\""
Jul 23 12:01:48.653530 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.653500    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-591d5c\"/\"default-dockercfg-lkxxl\""
Jul 23 12:01:48.654360 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.654340    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:01:48.719448 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.719662 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.719662 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719510    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfrf5\" (UniqueName: \"kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.719662 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719610    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.719662 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719650    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.719858 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.719682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821078 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821276 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821276 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821276 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821154    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821276 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfrf5\" (UniqueName: \"kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821276 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821219    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821602 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821560    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821659 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.821659 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.821630    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.823726 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.823702    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.824001 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.823982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.829938 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.829909    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfrf5\" (UniqueName: \"kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5\") pod \"test-model-591d5c-kserve-7d7fc4f8dc-tgsb2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") " pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:48.944441 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:48.944346    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:49.088677 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:49.088650    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:01:49.090832 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:01:49.090800    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f93ec8d_cdf7_47a2_ac58_23b29253bcc2.slice/crio-5b487963401eafb366ebe7265839ad3147bf3ca1a10d72845cafef028994fb94 WatchSource:0}: Error finding container 5b487963401eafb366ebe7265839ad3147bf3ca1a10d72845cafef028994fb94: Status 404 returned error can't find the container with id 5b487963401eafb366ebe7265839ad3147bf3ca1a10d72845cafef028994fb94
Jul 23 12:01:49.439829 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:49.439791    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" event={"ID":"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2","Type":"ContainerStarted","Data":"b3fa4cbff7b79b87c8bdf82400306c2555d919382bb4e96e0d53eb0271bba788"}
Jul 23 12:01:49.440024 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:49.439839    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" event={"ID":"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2","Type":"ContainerStarted","Data":"5b487963401eafb366ebe7265839ad3147bf3ca1a10d72845cafef028994fb94"}
Jul 23 12:01:49.457348 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:49.457287    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" podStartSLOduration=1.4572727460000001 podStartE2EDuration="1.457272746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:01:49.456567367 +0000 UTC m=+2123.742033134" watchObservedRunningTime="2026-07-23 12:01:49.457272746 +0000 UTC m=+2123.742738511"
Jul 23 12:01:52.235113 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:52.235077    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:01:58.945480 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:58.945435    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:58.945915 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:58.945513    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:58.958583 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:58.958558    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:01:59.500300 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:01:59.500266    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:02:10.131639 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:10.131605    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:02:29.530071 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:29.529989    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:02:33.631529 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:33.631496    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:02:40.091425 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.091391    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:02:40.091870 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.091642    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerName="maas-api" containerID="cri-o://0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07" gracePeriod=30
Jul 23 12:02:40.248059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.248016    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.48:8443/health\": dial tcp 10.132.0.48:8443: connect: connection refused"
Jul 23 12:02:40.351594 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.351532    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:02:40.504262 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.504218    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d\") pod \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") "
Jul 23 12:02:40.504448 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.504320    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") pod \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") "
Jul 23 12:02:40.504448 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.504364    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca\") pod \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\" (UID: \"486b5e6f-2e00-4422-b1c8-c69f1325eaa1\") "
Jul 23 12:02:40.504785 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.504760    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "486b5e6f-2e00-4422-b1c8-c69f1325eaa1" (UID: "486b5e6f-2e00-4422-b1c8-c69f1325eaa1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:02:40.506524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.506502    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "486b5e6f-2e00-4422-b1c8-c69f1325eaa1" (UID: "486b5e6f-2e00-4422-b1c8-c69f1325eaa1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:02:40.506524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.506510    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d" (OuterVolumeSpecName: "kube-api-access-sfk6d") pod "486b5e6f-2e00-4422-b1c8-c69f1325eaa1" (UID: "486b5e6f-2e00-4422-b1c8-c69f1325eaa1"). InnerVolumeSpecName "kube-api-access-sfk6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:02:40.606050 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.605957    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:40.606050 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.605989    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:40.606050 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.606002    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfk6d\" (UniqueName: \"kubernetes.io/projected/486b5e6f-2e00-4422-b1c8-c69f1325eaa1-kube-api-access-sfk6d\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:40.685926 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.685890    2574 generic.go:356] "Generic (PLEG): container finished" podID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerID="0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07" exitCode=0
Jul 23 12:02:40.686089 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.685936    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"
Jul 23 12:02:40.686089 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.685946    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" event={"ID":"486b5e6f-2e00-4422-b1c8-c69f1325eaa1","Type":"ContainerDied","Data":"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"}
Jul 23 12:02:40.686089 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.685988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654" event={"ID":"486b5e6f-2e00-4422-b1c8-c69f1325eaa1","Type":"ContainerDied","Data":"6f73e44281d8697df52360f33b87a0cb850968f92feb80fc5e7643495f07e1aa"}
Jul 23 12:02:40.686089 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.686009    2574 scope.go:117] "RemoveContainer" containerID="0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"
Jul 23 12:02:40.695341 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.695323    2574 scope.go:117] "RemoveContainer" containerID="0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"
Jul 23 12:02:40.695652 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:02:40.695634    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07\": container with ID starting with 0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07 not found: ID does not exist" containerID="0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"
Jul 23 12:02:40.695718 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.695662    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07"} err="failed to get container status \"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07\": rpc error: code = NotFound desc = could not find container \"0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07\": container with ID starting with 0923ffd7ae48b31f414d074e793e053dbb01f17d7de96c0b3f32ac594b210d07 not found: ID does not exist"
Jul 23 12:02:40.707753 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.707730    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:02:40.710688 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:40.710656    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-f8adc6-54c9bcb9c-mr654"]
Jul 23 12:02:42.298625 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:42.298591    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" path="/var/lib/kubelet/pods/486b5e6f-2e00-4422-b1c8-c69f1325eaa1/volumes"
Jul 23 12:02:44.550417 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.550354    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" podUID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" containerName="main" containerID="cri-o://b3fa4cbff7b79b87c8bdf82400306c2555d919382bb4e96e0d53eb0271bba788" gracePeriod=45
Jul 23 12:02:44.708415 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.708381    2574 generic.go:356] "Generic (PLEG): container finished" podID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" containerID="b3fa4cbff7b79b87c8bdf82400306c2555d919382bb4e96e0d53eb0271bba788" exitCode=0
Jul 23 12:02:44.708596 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.708452    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" event={"ID":"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2","Type":"ContainerDied","Data":"b3fa4cbff7b79b87c8bdf82400306c2555d919382bb4e96e0d53eb0271bba788"}
Jul 23 12:02:44.807785 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.807729    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:02:44.948273 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948238    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948288    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948320    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfrf5\" (UniqueName: \"kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948421    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948450    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948489    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs\") pod \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\" (UID: \"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2\") "
Jul 23 12:02:44.948782 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:02:44.948782 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948669    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home" (OuterVolumeSpecName: "home") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:02:44.948882 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948789    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:02:44.948882 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948846    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:44.948882 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.948866    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:44.950988 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.950961    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5" (OuterVolumeSpecName: "kube-api-access-pfrf5") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "kube-api-access-pfrf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:02:44.950988 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.950979    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:02:44.951130 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:44.950985    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm" (OuterVolumeSpecName: "dshm") pod "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" (UID: "3f93ec8d-cdf7-47a2-ac58-23b29253bcc2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:02:45.049349 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.049311    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfrf5\" (UniqueName: \"kubernetes.io/projected/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-kube-api-access-pfrf5\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:45.049349 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.049343    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:45.049349 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.049354    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:45.049616 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.049364    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:02:45.714773 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.714741    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"
Jul 23 12:02:45.715263 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.714738    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2" event={"ID":"3f93ec8d-cdf7-47a2-ac58-23b29253bcc2","Type":"ContainerDied","Data":"5b487963401eafb366ebe7265839ad3147bf3ca1a10d72845cafef028994fb94"}
Jul 23 12:02:45.715263 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.714871    2574 scope.go:117] "RemoveContainer" containerID="b3fa4cbff7b79b87c8bdf82400306c2555d919382bb4e96e0d53eb0271bba788"
Jul 23 12:02:45.736152 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.736126    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:02:45.739524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:45.739497    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-591d5c/test-model-591d5c-kserve-7d7fc4f8dc-tgsb2"]
Jul 23 12:02:46.298573 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:46.298541    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" path="/var/lib/kubelet/pods/3f93ec8d-cdf7-47a2-ac58-23b29253bcc2/volumes"
Jul 23 12:02:49.240638 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:02:49.240604    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:03:02.217485 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.217430    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:03:02.218091 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.217720    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" podUID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" containerName="maas-api" containerID="cri-o://b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1" gracePeriod=30
Jul 23 12:03:02.670756 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.670731    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:03:02.712086 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.712058    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca\") pod \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") "
Jul 23 12:03:02.712272 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.712121    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5r6c\" (UniqueName: \"kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c\") pod \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") "
Jul 23 12:03:02.712272 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.712149    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") pod \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\" (UID: \"9332080e-cce8-4e4b-aa5a-d545715dc7d7\") "
Jul 23 12:03:02.712528 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.712501    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9332080e-cce8-4e4b-aa5a-d545715dc7d7" (UID: "9332080e-cce8-4e4b-aa5a-d545715dc7d7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:03:02.714480 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.714430    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9332080e-cce8-4e4b-aa5a-d545715dc7d7" (UID: "9332080e-cce8-4e4b-aa5a-d545715dc7d7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:03:02.714587 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.714560    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c" (OuterVolumeSpecName: "kube-api-access-g5r6c") pod "9332080e-cce8-4e4b-aa5a-d545715dc7d7" (UID: "9332080e-cce8-4e4b-aa5a-d545715dc7d7"). InnerVolumeSpecName "kube-api-access-g5r6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:03:02.797213 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.797124    2574 generic.go:356] "Generic (PLEG): container finished" podID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" containerID="b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1" exitCode=0
Jul 23 12:03:02.797213 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.797172    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"
Jul 23 12:03:02.797403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.797210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" event={"ID":"9332080e-cce8-4e4b-aa5a-d545715dc7d7","Type":"ContainerDied","Data":"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"}
Jul 23 12:03:02.797403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.797266    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96" event={"ID":"9332080e-cce8-4e4b-aa5a-d545715dc7d7","Type":"ContainerDied","Data":"a40ba453ba62114b629923ee5fd4db90a7f1575c7961584d600a8126e3b1dffd"}
Jul 23 12:03:02.797403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.797289    2574 scope.go:117] "RemoveContainer" containerID="b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"
Jul 23 12:03:02.807135 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.807114    2574 scope.go:117] "RemoveContainer" containerID="b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"
Jul 23 12:03:02.807408 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:03:02.807391    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1\": container with ID starting with b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1 not found: ID does not exist" containerID="b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"
Jul 23 12:03:02.807501 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.807416    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1"} err="failed to get container status \"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1\": rpc error: code = NotFound desc = could not find container \"b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1\": container with ID starting with b2ae7566fa3e6cbd777b1867e81c94495a5ba2afba561a1bc8f8ac371814f2d1 not found: ID does not exist"
Jul 23 12:03:02.813107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.813083    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9332080e-cce8-4e4b-aa5a-d545715dc7d7-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:02.813178 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.813109    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5r6c\" (UniqueName: \"kubernetes.io/projected/9332080e-cce8-4e4b-aa5a-d545715dc7d7-kube-api-access-g5r6c\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:02.813178 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.813119    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9332080e-cce8-4e4b-aa5a-d545715dc7d7-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:02.819910 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.819887    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:03:02.822939 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:02.822915    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-591d5c-6bbf897947-cdp96"]
Jul 23 12:03:04.298362 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:04.298325    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" path="/var/lib/kubelet/pods/9332080e-cce8-4e4b-aa5a-d545715dc7d7/volumes"
Jul 23 12:03:05.141202 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:05.141160    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:03:17.543706 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.543667    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:17.545252 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545223    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerName="maas-api"
Jul 23 12:03:17.545414 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545403    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerName="maas-api"
Jul 23 12:03:17.545604 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545560    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" containerName="maas-api"
Jul 23 12:03:17.545710 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545698    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" containerName="maas-api"
Jul 23 12:03:17.545800 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545790    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" containerName="main"
Jul 23 12:03:17.545870 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.545862    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" containerName="main"
Jul 23 12:03:17.546106 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.546093    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9332080e-cce8-4e4b-aa5a-d545715dc7d7" containerName="maas-api"
Jul 23 12:03:17.546231 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.546210    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f93ec8d-cdf7-47a2-ac58-23b29253bcc2" containerName="main"
Jul 23 12:03:17.546355 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.546236    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="486b5e6f-2e00-4422-b1c8-c69f1325eaa1" containerName="maas-api"
Jul 23 12:03:17.550121 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.550098    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.553744 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.553713    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-43f746-serving-cert\""
Jul 23 12:03:17.556152 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.556129    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:17.644439 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.644400    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.644657 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.644511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.644657 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.644543    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7jk\" (UniqueName: \"kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.745475 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.745432    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.745650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.745522    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.745650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.745548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7jk\" (UniqueName: \"kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.746269 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.746240    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.748232 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.748197    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.754133 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.754110    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7jk\" (UniqueName: \"kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk\") pod \"maas-api-e2e-api-a-43f746-5b67fc5c86-996x8\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:17.862202 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:17.862162    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:18.001037 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.001012    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:18.003329 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:03:18.003303    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41429f99_6a5f_476d_82a3_91bfd18aca3a.slice/crio-1d828574e51c97fd2cc26470b35fac8e61ce405ec851e70138c3c86060f50455 WatchSource:0}: Error finding container 1d828574e51c97fd2cc26470b35fac8e61ce405ec851e70138c3c86060f50455: Status 404 returned error can't find the container with id 1d828574e51c97fd2cc26470b35fac8e61ce405ec851e70138c3c86060f50455
Jul 23 12:03:18.005031 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.005013    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 12:03:18.877873 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.877837    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" event={"ID":"41429f99-6a5f-476d-82a3-91bfd18aca3a","Type":"ContainerStarted","Data":"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"}
Jul 23 12:03:18.878360 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.877889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" event={"ID":"41429f99-6a5f-476d-82a3-91bfd18aca3a","Type":"ContainerStarted","Data":"1d828574e51c97fd2cc26470b35fac8e61ce405ec851e70138c3c86060f50455"}
Jul 23 12:03:18.878360 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.877956    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:18.896396 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:18.896347    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" podStartSLOduration=1.390118373 podStartE2EDuration="1.896330917s" totalImagesPullingTime="506.212544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:03:18.895389608 +0000 UTC m=+2213.180855373" watchObservedRunningTime="2026-07-23 12:03:18.896330917 +0000 UTC m=+2213.181796682"
Jul 23 12:03:24.888089 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:24.888056    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:25.332381 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.332355    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:03:25.746247 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.746158    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:03:25.750533 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.750508    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.753077 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.753056    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f1f99f-serving-cert\""
Jul 23 12:03:25.762266 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.762194    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:03:25.816105 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.816065    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.816105 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.816111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qqhc\" (UniqueName: \"kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.816336 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.816180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.916910 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.916876    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.917398 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.916934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qqhc\" (UniqueName: \"kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.917398 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.917032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.917398 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:03:25.917230    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-serving-cert: secret "maas-api-e2e-api-b-f1f99f-serving-cert" not found
Jul 23 12:03:25.917398 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:03:25.917297    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls podName:ea8b6c4e-b974-46b0-82c5-f3603d9d0d43 nodeName:}" failed. No retries permitted until 2026-07-23 12:03:26.417275446 +0000 UTC m=+2220.702741196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls") pod "maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" (UID: "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43") : secret "maas-api-e2e-api-b-f1f99f-serving-cert" not found
Jul 23 12:03:25.917642 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.917592    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:25.926925 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:25.926898    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qqhc\" (UniqueName: \"kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:26.422948 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:26.422909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:26.425510 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:26.425483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") pod \"maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:26.665422 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:26.665385    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:26.806566 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:26.806543    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:03:26.808372 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:03:26.808346    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8b6c4e_b974_46b0_82c5_f3603d9d0d43.slice/crio-afddc36988c9e1c2473b1f36176885f2760068eeac441d094b3493c71942a2ff WatchSource:0}: Error finding container afddc36988c9e1c2473b1f36176885f2760068eeac441d094b3493c71942a2ff: Status 404 returned error can't find the container with id afddc36988c9e1c2473b1f36176885f2760068eeac441d094b3493c71942a2ff
Jul 23 12:03:26.916927 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:26.916885    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" event={"ID":"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43","Type":"ContainerStarted","Data":"afddc36988c9e1c2473b1f36176885f2760068eeac441d094b3493c71942a2ff"}
Jul 23 12:03:27.923782 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:27.923749    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" event={"ID":"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43","Type":"ContainerStarted","Data":"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"}
Jul 23 12:03:27.924196 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:27.923858    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:27.940845 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:27.940787    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" podStartSLOduration=2.591569061 podStartE2EDuration="2.940767866s" totalImagesPullingTime="349.198805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:03:27.940692758 +0000 UTC m=+2222.226158515" watchObservedRunningTime="2026-07-23 12:03:27.940767866 +0000 UTC m=+2222.226233631"
Jul 23 12:03:33.933621 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:33.933587    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:03:41.544486 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:41.539454    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:03:53.259380 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.259297    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:53.259869 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.259586    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" podUID="41429f99-6a5f-476d-82a3-91bfd18aca3a" containerName="maas-api" containerID="cri-o://33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90" gracePeriod=30
Jul 23 12:03:53.515878 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.515819    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:53.672221 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.672190    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw7jk\" (UniqueName: \"kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk\") pod \"41429f99-6a5f-476d-82a3-91bfd18aca3a\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") "
Jul 23 12:03:53.672370 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.672236    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca\") pod \"41429f99-6a5f-476d-82a3-91bfd18aca3a\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") "
Jul 23 12:03:53.672370 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.672276    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls\") pod \"41429f99-6a5f-476d-82a3-91bfd18aca3a\" (UID: \"41429f99-6a5f-476d-82a3-91bfd18aca3a\") "
Jul 23 12:03:53.672679 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.672644    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41429f99-6a5f-476d-82a3-91bfd18aca3a" (UID: "41429f99-6a5f-476d-82a3-91bfd18aca3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:03:53.674588 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.674562    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41429f99-6a5f-476d-82a3-91bfd18aca3a" (UID: "41429f99-6a5f-476d-82a3-91bfd18aca3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:03:53.674588 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.674580    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk" (OuterVolumeSpecName: "kube-api-access-xw7jk") pod "41429f99-6a5f-476d-82a3-91bfd18aca3a" (UID: "41429f99-6a5f-476d-82a3-91bfd18aca3a"). InnerVolumeSpecName "kube-api-access-xw7jk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:03:53.773787 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.773698    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw7jk\" (UniqueName: \"kubernetes.io/projected/41429f99-6a5f-476d-82a3-91bfd18aca3a-kube-api-access-xw7jk\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:53.773787 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.773728    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41429f99-6a5f-476d-82a3-91bfd18aca3a-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:53.773787 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:53.773738    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41429f99-6a5f-476d-82a3-91bfd18aca3a-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:03:54.049260 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.049168    2574 generic.go:356] "Generic (PLEG): container finished" podID="41429f99-6a5f-476d-82a3-91bfd18aca3a" containerID="33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90" exitCode=0
Jul 23 12:03:54.049260 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.049218    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"
Jul 23 12:03:54.049524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.049266    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" event={"ID":"41429f99-6a5f-476d-82a3-91bfd18aca3a","Type":"ContainerDied","Data":"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"}
Jul 23 12:03:54.049524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.049326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8" event={"ID":"41429f99-6a5f-476d-82a3-91bfd18aca3a","Type":"ContainerDied","Data":"1d828574e51c97fd2cc26470b35fac8e61ce405ec851e70138c3c86060f50455"}
Jul 23 12:03:54.049524 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.049358    2574 scope.go:117] "RemoveContainer" containerID="33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"
Jul 23 12:03:54.060066 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.060049    2574 scope.go:117] "RemoveContainer" containerID="33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"
Jul 23 12:03:54.060309 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:03:54.060284    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90\": container with ID starting with 33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90 not found: ID does not exist" containerID="33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"
Jul 23 12:03:54.060353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.060318    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90"} err="failed to get container status \"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90\": rpc error: code = NotFound desc = could not find container \"33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90\": container with ID starting with 33f6d0f3a94a9d9d8efad57aec26d81e3e75f27487bdfbfcb0dc76b2bc7ddc90 not found: ID does not exist"
Jul 23 12:03:54.073165 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.073141    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:54.079397 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.079374    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-43f746-5b67fc5c86-996x8"]
Jul 23 12:03:54.300491 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:54.300397    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41429f99-6a5f-476d-82a3-91bfd18aca3a" path="/var/lib/kubelet/pods/41429f99-6a5f-476d-82a3-91bfd18aca3a/volumes"
Jul 23 12:03:57.332659 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:03:57.332626    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:04:20.487765 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.487732    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:04:20.488339 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.487956    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" podUID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" containerName="maas-api" containerID="cri-o://aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375" gracePeriod=30
Jul 23 12:04:20.739062 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.739002    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:04:20.816210 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.816178    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") pod \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") "
Jul 23 12:04:20.816396 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.816262    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qqhc\" (UniqueName: \"kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc\") pod \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") "
Jul 23 12:04:20.816396 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.816317    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca\") pod \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\" (UID: \"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43\") "
Jul 23 12:04:20.816745 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.816713    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" (UID: "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:04:20.818554 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.818532    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc" (OuterVolumeSpecName: "kube-api-access-7qqhc") pod "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" (UID: "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43"). InnerVolumeSpecName "kube-api-access-7qqhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:04:20.818675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.818656    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" (UID: "ea8b6c4e-b974-46b0-82c5-f3603d9d0d43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:04:20.917939 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.917901    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:04:20.917939 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.917935    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:04:20.917939 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:20.917944    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qqhc\" (UniqueName: \"kubernetes.io/projected/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43-kube-api-access-7qqhc\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:04:21.176487 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.176425    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" containerID="aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375" exitCode=0
Jul 23 12:04:21.176675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.176494    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"
Jul 23 12:04:21.176675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.176507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" event={"ID":"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43","Type":"ContainerDied","Data":"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"}
Jul 23 12:04:21.176675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.176560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6" event={"ID":"ea8b6c4e-b974-46b0-82c5-f3603d9d0d43","Type":"ContainerDied","Data":"afddc36988c9e1c2473b1f36176885f2760068eeac441d094b3493c71942a2ff"}
Jul 23 12:04:21.176675 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.176582    2574 scope.go:117] "RemoveContainer" containerID="aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"
Jul 23 12:04:21.185899 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.185884    2574 scope.go:117] "RemoveContainer" containerID="aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"
Jul 23 12:04:21.186159 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:04:21.186140    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375\": container with ID starting with aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375 not found: ID does not exist" containerID="aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"
Jul 23 12:04:21.186208 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.186169    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375"} err="failed to get container status \"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375\": rpc error: code = NotFound desc = could not find container \"aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375\": container with ID starting with aed0e40ef3250ccb0f54b31542fc9ec7f187a51856e3ed6ff20c37d1459b7375 not found: ID does not exist"
Jul 23 12:04:21.198778 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.198753    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:04:21.202708 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:21.202687    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1f99f-55497f64cd-9pfh6"]
Jul 23 12:04:22.297628 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:22.297586    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" path="/var/lib/kubelet/pods/ea8b6c4e-b974-46b0-82c5-f3603d9d0d43/volumes"
Jul 23 12:04:25.239989 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:25.239946    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:04:36.150083 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150039    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:04:36.150603 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150586    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41429f99-6a5f-476d-82a3-91bfd18aca3a" containerName="maas-api"
Jul 23 12:04:36.150603 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150604    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="41429f99-6a5f-476d-82a3-91bfd18aca3a" containerName="maas-api"
Jul 23 12:04:36.150720 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150621    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" containerName="maas-api"
Jul 23 12:04:36.150720 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150627    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" containerName="maas-api"
Jul 23 12:04:36.150720 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150704    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="41429f99-6a5f-476d-82a3-91bfd18aca3a" containerName="maas-api"
Jul 23 12:04:36.150720 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.150714    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8b6c4e-b974-46b0-82c5-f3603d9d0d43" containerName="maas-api"
Jul 23 12:04:36.153995 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.153977    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.158198 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.158177    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7cc2e5-serving-cert\""
Jul 23 12:04:36.170039 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.169969    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:04:36.260345 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.260304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.260345 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.260347    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfkq\" (UniqueName: \"kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.260640 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.260496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.361892 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.361850    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.362109 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.361919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.362109 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.362028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfkq\" (UniqueName: \"kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.362232 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:04:36.362033    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-serving-cert: secret "maas-api-e2e-shared-a-7cc2e5-serving-cert" not found
Jul 23 12:04:36.362232 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:04:36.362193    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls podName:b418109a-7d5d-445d-b6e1-312f4b2a4ad6 nodeName:}" failed. No retries permitted until 2026-07-23 12:04:36.862176043 +0000 UTC m=+2291.147641790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls") pod "maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" (UID: "b418109a-7d5d-445d-b6e1-312f4b2a4ad6") : secret "maas-api-e2e-shared-a-7cc2e5-serving-cert" not found
Jul 23 12:04:36.362550 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.362532    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.373340 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.373305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfkq\" (UniqueName: \"kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.866423 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.866381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:36.869116 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:36.869079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") pod \"maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:37.064218 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:37.064188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:37.205750 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:37.205719    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:04:37.207705 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:04:37.207676    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb418109a_7d5d_445d_b6e1_312f4b2a4ad6.slice/crio-949f2852a2233d2c8d7cc7fa7375281855cdf32f3c363baa60dd493d1a391d5c WatchSource:0}: Error finding container 949f2852a2233d2c8d7cc7fa7375281855cdf32f3c363baa60dd493d1a391d5c: Status 404 returned error can't find the container with id 949f2852a2233d2c8d7cc7fa7375281855cdf32f3c363baa60dd493d1a391d5c
Jul 23 12:04:37.253447 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:37.253408    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" event={"ID":"b418109a-7d5d-445d-b6e1-312f4b2a4ad6","Type":"ContainerStarted","Data":"949f2852a2233d2c8d7cc7fa7375281855cdf32f3c363baa60dd493d1a391d5c"}
Jul 23 12:04:38.259835 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:38.259744    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" event={"ID":"b418109a-7d5d-445d-b6e1-312f4b2a4ad6","Type":"ContainerStarted","Data":"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"}
Jul 23 12:04:38.260213 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:38.259874    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:38.278678 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:38.278620    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" podStartSLOduration=1.6162482040000001 podStartE2EDuration="2.27860296s" totalImagesPullingTime="662.354756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:04:38.277614274 +0000 UTC m=+2292.563080040" watchObservedRunningTime="2026-07-23 12:04:38.27860296 +0000 UTC m=+2292.564068726"
Jul 23 12:04:44.046970 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.046940    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:04:44.186169 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.186131    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:04:44.190490 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.190451    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.193199 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.193172    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-77f87c-serving-cert\""
Jul 23 12:04:44.203191 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.203121    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:04:44.235488 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.235431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.235649 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.235544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.235703 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.235659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zlvp\" (UniqueName: \"kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.269612 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.269585    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:04:44.336290 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.336257    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.336486 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.336329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.336567 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:04:44.336511    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-serving-cert: secret "maas-api-e2e-shared-b-77f87c-serving-cert" not found
Jul 23 12:04:44.336640 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.336583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zlvp\" (UniqueName: \"kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.336640 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:04:44.336604    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls podName:3d3764de-a179-403c-9474-d436ef99ae01 nodeName:}" failed. No retries permitted until 2026-07-23 12:04:44.836580747 +0000 UTC m=+2299.122046494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls") pod "maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" (UID: "3d3764de-a179-403c-9474-d436ef99ae01") : secret "maas-api-e2e-shared-b-77f87c-serving-cert" not found
Jul 23 12:04:44.336993 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.336972    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.351879 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.351855    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zlvp\" (UniqueName: \"kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.840350 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.840314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:44.842899 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:44.842880    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") pod \"maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:45.102965 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:45.102873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:45.241413 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:45.241380    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:04:45.242205 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:04:45.242176    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3764de_a179_403c_9474_d436ef99ae01.slice/crio-266c812f9be6c5243e52e1b0bae340e9308534d3e84d2800db6331e5e068b5e8 WatchSource:0}: Error finding container 266c812f9be6c5243e52e1b0bae340e9308534d3e84d2800db6331e5e068b5e8: Status 404 returned error can't find the container with id 266c812f9be6c5243e52e1b0bae340e9308534d3e84d2800db6331e5e068b5e8
Jul 23 12:04:45.293531 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:45.293500    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" event={"ID":"3d3764de-a179-403c-9474-d436ef99ae01","Type":"ContainerStarted","Data":"266c812f9be6c5243e52e1b0bae340e9308534d3e84d2800db6331e5e068b5e8"}
Jul 23 12:04:46.299548 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:46.299520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" event={"ID":"3d3764de-a179-403c-9474-d436ef99ae01","Type":"ContainerStarted","Data":"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"}
Jul 23 12:04:46.299920 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:46.299677    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:46.318481 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:46.318416    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" podStartSLOduration=1.375618727 podStartE2EDuration="2.318396674s" totalImagesPullingTime="942.777947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:04:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:04:46.317875246 +0000 UTC m=+2300.603341012" watchObservedRunningTime="2026-07-23 12:04:46.318396674 +0000 UTC m=+2300.603862440"
Jul 23 12:04:52.310926 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:52.310893    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:04:58.338683 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:04:58.338648    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:05:12.098972 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.098937    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:05:12.102533 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.102514    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.106840 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.106820    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"openshift-service-ca.crt\""
Jul 23 12:05:12.106976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.106844    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"default-dockercfg-6scws\""
Jul 23 12:05:12.106976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.106916    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"auth-test-model-7cc2e5-kserve-self-signed-certs\""
Jul 23 12:05:12.107091 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.107019    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"kube-root-ca.crt\""
Jul 23 12:05:12.120137 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.120113    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:05:12.192897 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.192857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.193093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.192927    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fstv\" (UniqueName: \"kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.193093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.192963    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.193093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.193007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.193093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.193054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.193093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.193077    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294422 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294634 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294634 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294454    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294634 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fstv\" (UniqueName: \"kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294634 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294555    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294634 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294618    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.294931 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.295114 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.294955    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.295114 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.295032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.296952 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.296931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.297410 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.297390    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.304173 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.304150    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fstv\" (UniqueName: \"kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv\") pod \"auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") " pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.411408 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.411324    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:12.554491 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:12.549573    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:05:13.427379 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:13.427344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" event={"ID":"a33cca46-fbb5-41d5-8224-b340bfa7fe51","Type":"ContainerStarted","Data":"ce439c6db1e55c37b440d0ab3e2513bdc6b9a402a92f928c1f1e738c908e636c"}
Jul 23 12:05:13.427379 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:13.427383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" event={"ID":"a33cca46-fbb5-41d5-8224-b340bfa7fe51","Type":"ContainerStarted","Data":"915e68904b45888a9a9b57e210fe2d5ef5f2f3c46b433768b99d457327cb759c"}
Jul 23 12:05:13.445035 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:13.444984    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" podStartSLOduration=1.444968758 podStartE2EDuration="1.444968758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:05:13.443784719 +0000 UTC m=+2327.729250485" watchObservedRunningTime="2026-07-23 12:05:13.444968758 +0000 UTC m=+2327.730434523"
Jul 23 12:05:15.043650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:15.043615    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:05:22.412501 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:22.412401    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:22.412501 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:22.412442    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:22.424966 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:22.424933    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:22.482200 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:22.482165    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:05:34.642124 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:34.642080    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:05:53.550698 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:05:53.550662    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:06:11.442158 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:11.442116    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:06:26.363737 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:26.363706    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:06:26.370343 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:26.370321    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:06:26.376051 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:26.376027    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:06:26.382331 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:26.382310    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:06:31.636433 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:31.636399    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:06:50.243009 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:06:50.242931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:07:04.953442 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:04.953398    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:07:22.134265 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:22.134227    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:07:38.031455 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:38.031417    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:07:41.141095 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:41.141064    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:07:53.051922 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.051876    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" podUID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" containerName="main" containerID="cri-o://ce439c6db1e55c37b440d0ab3e2513bdc6b9a402a92f928c1f1e738c908e636c" gracePeriod=45
Jul 23 12:07:53.209247 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.209215    2574 generic.go:356] "Generic (PLEG): container finished" podID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" containerID="ce439c6db1e55c37b440d0ab3e2513bdc6b9a402a92f928c1f1e738c908e636c" exitCode=0
Jul 23 12:07:53.209443 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.209298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" event={"ID":"a33cca46-fbb5-41d5-8224-b340bfa7fe51","Type":"ContainerDied","Data":"ce439c6db1e55c37b440d0ab3e2513bdc6b9a402a92f928c1f1e738c908e636c"}
Jul 23 12:07:53.311707 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.311642    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:07:53.386534 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386436    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.386534 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386514    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.386788 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.386788 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386738    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.386788 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386757    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:07:53.386788 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386773    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fstv\" (UniqueName: \"kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.387000 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386791    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache" (OuterVolumeSpecName: "model-cache") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:07:53.387000 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386811    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm\") pod \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\" (UID: \"a33cca46-fbb5-41d5-8224-b340bfa7fe51\") "
Jul 23 12:07:53.387000 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.386915    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home" (OuterVolumeSpecName: "home") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:07:53.387144 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.387128    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:53.387189 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.387152    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:53.387189 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.387165    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:53.389177 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.389144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:07:53.389309 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.389177    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv" (OuterVolumeSpecName: "kube-api-access-8fstv") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "kube-api-access-8fstv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:07:53.389309 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.389229    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm" (OuterVolumeSpecName: "dshm") pod "a33cca46-fbb5-41d5-8224-b340bfa7fe51" (UID: "a33cca46-fbb5-41d5-8224-b340bfa7fe51"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:07:53.488660 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.488629    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fstv\" (UniqueName: \"kubernetes.io/projected/a33cca46-fbb5-41d5-8224-b340bfa7fe51-kube-api-access-8fstv\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:53.488660 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.488654    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a33cca46-fbb5-41d5-8224-b340bfa7fe51-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:53.488660 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:53.488665    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a33cca46-fbb5-41d5-8224-b340bfa7fe51-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:07:54.215290 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.215261    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"
Jul 23 12:07:54.215290 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.215286    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq" event={"ID":"a33cca46-fbb5-41d5-8224-b340bfa7fe51","Type":"ContainerDied","Data":"915e68904b45888a9a9b57e210fe2d5ef5f2f3c46b433768b99d457327cb759c"}
Jul 23 12:07:54.215819 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.215332    2574 scope.go:117] "RemoveContainer" containerID="ce439c6db1e55c37b440d0ab3e2513bdc6b9a402a92f928c1f1e738c908e636c"
Jul 23 12:07:54.241448 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.241411    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:07:54.246152 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.246130    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/auth-test-model-7cc2e5-kserve-5d6d8764f5-dkdmq"]
Jul 23 12:07:54.298099 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:54.298067    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" path="/var/lib/kubelet/pods/a33cca46-fbb5-41d5-8224-b340bfa7fe51/volumes"
Jul 23 12:07:56.539658 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:07:56.539626    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:08:06.034079 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:06.034039    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:08:33.936137 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.936098    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:08:33.936683 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.936510    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" containerName="main"
Jul 23 12:08:33.936683 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.936525    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" containerName="main"
Jul 23 12:08:33.936683 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.936620    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33cca46-fbb5-41d5-8224-b340bfa7fe51" containerName="main"
Jul 23 12:08:33.939536 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.939515    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:33.943436 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.943415    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-77f87c\"/\"openshift-service-ca.crt\""
Jul 23 12:08:33.943564 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.943419    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-77f87c\"/\"default-dockercfg-xx7rr\""
Jul 23 12:08:33.943564 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.943418    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-77f87c\"/\"sub-test-model-77f87c-kserve-self-signed-certs\""
Jul 23 12:08:33.943717 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.943704    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-77f87c\"/\"kube-root-ca.crt\""
Jul 23 12:08:33.949491 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:33.949454    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:08:34.048928 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.048896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.049107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.048948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.049107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.048967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.049107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.048985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.049107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.049015    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.049107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.049056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d42lm\" (UniqueName: \"kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150424 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150387    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d42lm\" (UniqueName: \"kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150650 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.150852 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.151026 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.150996    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.151142 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.151034    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.151142 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.151070    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.152956 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.152925    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.153178 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.153160    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.158716 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.158697    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d42lm\" (UniqueName: \"kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm\") pod \"sub-test-model-77f87c-kserve-9f74dc779-ddkjc\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") " pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.249385 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.249307    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:34.387308 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.387283    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:08:34.389658 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:08:34.389624    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c2b196b_93f2_43f6_845b_0a74701e91ed.slice/crio-b7ee66f4f2cd17c373e4f4fd1b8d54e0f092e45d57739e0c6e289930bd00cbab WatchSource:0}: Error finding container b7ee66f4f2cd17c373e4f4fd1b8d54e0f092e45d57739e0c6e289930bd00cbab: Status 404 returned error can't find the container with id b7ee66f4f2cd17c373e4f4fd1b8d54e0f092e45d57739e0c6e289930bd00cbab
Jul 23 12:08:34.405991 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:34.405962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" event={"ID":"8c2b196b-93f2-43f6-845b-0a74701e91ed","Type":"ContainerStarted","Data":"b7ee66f4f2cd17c373e4f4fd1b8d54e0f092e45d57739e0c6e289930bd00cbab"}
Jul 23 12:08:35.412901 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:35.412858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" event={"ID":"8c2b196b-93f2-43f6-845b-0a74701e91ed","Type":"ContainerStarted","Data":"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"}
Jul 23 12:08:35.435277 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:35.435233    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" podStartSLOduration=2.435218347 podStartE2EDuration="2.435218347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:08:35.432870076 +0000 UTC m=+2529.718335841" watchObservedRunningTime="2026-07-23 12:08:35.435218347 +0000 UTC m=+2529.720684114"
Jul 23 12:08:37.732788 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:37.732758    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:08:44.250168 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:44.250136    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:44.250711 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:44.250213    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:44.262827 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:44.262802    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:44.466243 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:44.466210    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:08:59.739275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:08:59.739242    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:09:14.046976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:14.046942    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:09:21.047749 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:21.047705    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:09:29.833370 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.833335    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:09:29.837107 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.837082    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.840720 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.840699    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"default-dockercfg-6scws\""
Jul 23 12:09:29.840848 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.840724    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"kube-root-ca.crt\""
Jul 23 12:09:29.840848 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.840699    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"openshift-service-ca.crt\""
Jul 23 12:09:29.840848 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.840699    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7cc2e5\"/\"rate-test-model-7cc2e5-kserve-self-signed-certs\""
Jul 23 12:09:29.847867 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.847844    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:09:29.970109 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.970109 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.970364 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.970364 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.970364 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zv87\" (UniqueName: \"kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:29.970364 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:29.970289    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.071791 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.071751    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.071989 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.071861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.071989 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.071902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.071989 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.071958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.072165 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.072032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.072165 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.072051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zv87\" (UniqueName: \"kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.072334 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.072315    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.072401 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.072365    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.072401 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.072385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.074367 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.074344    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.074505 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.074441    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.080427 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.080407    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zv87\" (UniqueName: \"kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87\") pod \"rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") " pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.146953 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.146864    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:30.284642 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.284615    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:09:30.287164 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:09:30.287135    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a2e4ae8_89b2_4dca_9a94_5719e94b8f94.slice/crio-7f04b70de4547d8b09cb3e21fb138b1c1340e6b82a96ec15f9056050a19e9b58 WatchSource:0}: Error finding container 7f04b70de4547d8b09cb3e21fb138b1c1340e6b82a96ec15f9056050a19e9b58: Status 404 returned error can't find the container with id 7f04b70de4547d8b09cb3e21fb138b1c1340e6b82a96ec15f9056050a19e9b58
Jul 23 12:09:30.676610 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.676575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" event={"ID":"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94","Type":"ContainerStarted","Data":"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"}
Jul 23 12:09:30.676784 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.676617    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" event={"ID":"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94","Type":"ContainerStarted","Data":"7f04b70de4547d8b09cb3e21fb138b1c1340e6b82a96ec15f9056050a19e9b58"}
Jul 23 12:09:30.698275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:30.698225    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" podStartSLOduration=1.6982085260000002 podStartE2EDuration="1.698208526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:09:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:09:30.696349104 +0000 UTC m=+2584.981814882" watchObservedRunningTime="2026-07-23 12:09:30.698208526 +0000 UTC m=+2584.983674291"
Jul 23 12:09:33.136235 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:33.136201    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:09:36.070821 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.070759    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" podUID="8c2b196b-93f2-43f6-845b-0a74701e91ed" containerName="main" containerID="cri-o://d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616" gracePeriod=45
Jul 23 12:09:36.321169 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.321102    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:09:36.437973 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.437937    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.437973 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.437977    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.438218 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438012    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.438218 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438069    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.438218 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438149    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.438218 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438173    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d42lm\" (UniqueName: \"kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm\") pod \"8c2b196b-93f2-43f6-845b-0a74701e91ed\" (UID: \"8c2b196b-93f2-43f6-845b-0a74701e91ed\") "
Jul 23 12:09:36.438425 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:09:36.438425 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438304    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:09:36.438425 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438388    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home" (OuterVolumeSpecName: "home") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:09:36.438646 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438606    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.438646 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438627    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.438646 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.438641    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.440550 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.440525    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm" (OuterVolumeSpecName: "dshm") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:09:36.440974 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.440955    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:09:36.440974 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.440962    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm" (OuterVolumeSpecName: "kube-api-access-d42lm") pod "8c2b196b-93f2-43f6-845b-0a74701e91ed" (UID: "8c2b196b-93f2-43f6-845b-0a74701e91ed"). InnerVolumeSpecName "kube-api-access-d42lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:09:36.539823 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.539793    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2b196b-93f2-43f6-845b-0a74701e91ed-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.539823 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.539823    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d42lm\" (UniqueName: \"kubernetes.io/projected/8c2b196b-93f2-43f6-845b-0a74701e91ed-kube-api-access-d42lm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.540012 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.539837    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2b196b-93f2-43f6-845b-0a74701e91ed-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:09:36.706427 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.706392    2574 generic.go:356] "Generic (PLEG): container finished" podID="8c2b196b-93f2-43f6-845b-0a74701e91ed" containerID="d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616" exitCode=0
Jul 23 12:09:36.706632 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.706433    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"
Jul 23 12:09:36.706632 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.706439    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" event={"ID":"8c2b196b-93f2-43f6-845b-0a74701e91ed","Type":"ContainerDied","Data":"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"}
Jul 23 12:09:36.706632 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.706507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc" event={"ID":"8c2b196b-93f2-43f6-845b-0a74701e91ed","Type":"ContainerDied","Data":"b7ee66f4f2cd17c373e4f4fd1b8d54e0f092e45d57739e0c6e289930bd00cbab"}
Jul 23 12:09:36.706632 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.706525    2574 scope.go:117] "RemoveContainer" containerID="d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"
Jul 23 12:09:36.715965 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.715944    2574 scope.go:117] "RemoveContainer" containerID="d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"
Jul 23 12:09:36.716229 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:09:36.716209    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616\": container with ID starting with d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616 not found: ID does not exist" containerID="d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"
Jul 23 12:09:36.716287 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.716237    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616"} err="failed to get container status \"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616\": rpc error: code = NotFound desc = could not find container \"d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616\": container with ID starting with d2d9e741f993f9709c0a4bda06df09b4bf635d4a387ea4ca6129096fa3357616 not found: ID does not exist"
Jul 23 12:09:36.729120 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.729092    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:09:36.733603 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:36.733567    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-77f87c/sub-test-model-77f87c-kserve-9f74dc779-ddkjc"]
Jul 23 12:09:38.297990 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:38.297957    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c2b196b-93f2-43f6-845b-0a74701e91ed" path="/var/lib/kubelet/pods/8c2b196b-93f2-43f6-845b-0a74701e91ed/volumes"
Jul 23 12:09:40.147536 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:40.147497    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:40.148015 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:40.147558    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:40.159937 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:40.159912    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:40.737972 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:40.737941    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:09:54.140224 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:09:54.140192    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:10:25.839838 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:10:25.839800    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:10:41.245680 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:10:41.245635    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:10:48.460016 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:10:48.459981    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:11:02.540909 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:02.540873    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:11:03.481531 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.481489    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" podUID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" containerName="main" containerID="cri-o://129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7" gracePeriod=45
Jul 23 12:11:03.740500 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.740420    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:11:03.854561 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854530    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854758 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854593    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854758 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854641    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854758 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854719    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:11:03.854758 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854746    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854779    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854809    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zv87\" (UniqueName: \"kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87\") pod \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\" (UID: \"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94\") "
Jul 23 12:11:03.854976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854857    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home" (OuterVolumeSpecName: "home") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:11:03.854976 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.854964    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache" (OuterVolumeSpecName: "model-cache") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:11:03.855140 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.855126    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-model-cache\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:03.855140 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.855140    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tmp-dir\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:03.855241 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.855148    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-home\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:03.856905 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.856878    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm" (OuterVolumeSpecName: "dshm") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 12:11:03.857536 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.857505    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:11:03.857536 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.857530    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87" (OuterVolumeSpecName: "kube-api-access-4zv87") pod "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" (UID: "6a2e4ae8-89b2-4dca-9a94-5719e94b8f94"). InnerVolumeSpecName "kube-api-access-4zv87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:11:03.956095 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.956057    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-dshm\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:03.956095 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.956088    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-tls-certs\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:03.956095 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:03.956103    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zv87\" (UniqueName: \"kubernetes.io/projected/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94-kube-api-access-4zv87\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:04.126258 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.126221    2574 generic.go:356] "Generic (PLEG): container finished" podID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" containerID="129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7" exitCode=0
Jul 23 12:11:04.126484 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.126265    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"
Jul 23 12:11:04.126484 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.126301    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" event={"ID":"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94","Type":"ContainerDied","Data":"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"}
Jul 23 12:11:04.126484 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.126344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf" event={"ID":"6a2e4ae8-89b2-4dca-9a94-5719e94b8f94","Type":"ContainerDied","Data":"7f04b70de4547d8b09cb3e21fb138b1c1340e6b82a96ec15f9056050a19e9b58"}
Jul 23 12:11:04.126484 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.126361    2574 scope.go:117] "RemoveContainer" containerID="129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"
Jul 23 12:11:04.136231 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.136214    2574 scope.go:117] "RemoveContainer" containerID="129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"
Jul 23 12:11:04.136540 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:11:04.136521    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7\": container with ID starting with 129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7 not found: ID does not exist" containerID="129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"
Jul 23 12:11:04.136615 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.136551    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7"} err="failed to get container status \"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7\": rpc error: code = NotFound desc = could not find container \"129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7\": container with ID starting with 129b1e44ff2a1678b17997f0c4e11d9a331e5ba50df2a04781029ee9b9635be7 not found: ID does not exist"
Jul 23 12:11:04.151842 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.151816    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:11:04.156977 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.156956    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7cc2e5/rate-test-model-7cc2e5-kserve-84556c7fdc-2kxnf"]
Jul 23 12:11:04.299024 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:04.298989    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" path="/var/lib/kubelet/pods/6a2e4ae8-89b2-4dca-9a94-5719e94b8f94/volumes"
Jul 23 12:11:19.407001 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:19.406896    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:11:19.407597 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:19.407142    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" podUID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" containerName="maas-api" containerID="cri-o://79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970" gracePeriod=30
Jul 23 12:11:20.657692 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.657665    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:11:20.713115 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.713039    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptfkq\" (UniqueName: \"kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq\") pod \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") "
Jul 23 12:11:20.713115 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.713076    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") pod \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") "
Jul 23 12:11:20.713115 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.713099    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca\") pod \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\" (UID: \"b418109a-7d5d-445d-b6e1-312f4b2a4ad6\") "
Jul 23 12:11:20.713560 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.713534    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b418109a-7d5d-445d-b6e1-312f4b2a4ad6" (UID: "b418109a-7d5d-445d-b6e1-312f4b2a4ad6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:11:20.715303 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.715281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq" (OuterVolumeSpecName: "kube-api-access-ptfkq") pod "b418109a-7d5d-445d-b6e1-312f4b2a4ad6" (UID: "b418109a-7d5d-445d-b6e1-312f4b2a4ad6"). InnerVolumeSpecName "kube-api-access-ptfkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:11:20.715303 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.715286    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b418109a-7d5d-445d-b6e1-312f4b2a4ad6" (UID: "b418109a-7d5d-445d-b6e1-312f4b2a4ad6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:11:20.814395 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.814361    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptfkq\" (UniqueName: \"kubernetes.io/projected/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-kube-api-access-ptfkq\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:20.814395 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.814392    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:20.814395 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:20.814402    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b418109a-7d5d-445d-b6e1-312f4b2a4ad6-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:21.210557 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.210527    2574 generic.go:356] "Generic (PLEG): container finished" podID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" containerID="79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970" exitCode=0
Jul 23 12:11:21.210747 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.210574    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"
Jul 23 12:11:21.210747 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.210621    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" event={"ID":"b418109a-7d5d-445d-b6e1-312f4b2a4ad6","Type":"ContainerDied","Data":"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"}
Jul 23 12:11:21.210747 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.210698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d" event={"ID":"b418109a-7d5d-445d-b6e1-312f4b2a4ad6","Type":"ContainerDied","Data":"949f2852a2233d2c8d7cc7fa7375281855cdf32f3c363baa60dd493d1a391d5c"}
Jul 23 12:11:21.210747 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.210720    2574 scope.go:117] "RemoveContainer" containerID="79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"
Jul 23 12:11:21.219775 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.219758    2574 scope.go:117] "RemoveContainer" containerID="79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"
Jul 23 12:11:21.220045 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:11:21.220027    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970\": container with ID starting with 79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970 not found: ID does not exist" containerID="79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"
Jul 23 12:11:21.220095 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.220056    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970"} err="failed to get container status \"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970\": rpc error: code = NotFound desc = could not find container \"79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970\": container with ID starting with 79398bef2b383d15e1af2c525239933f123b2330a13991d052fef8fd44374970 not found: ID does not exist"
Jul 23 12:11:21.232717 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.232695    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:11:21.236591 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:21.236571    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7cc2e5-5f6d995474-dlz8d"]
Jul 23 12:11:22.299735 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:22.299699    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" path="/var/lib/kubelet/pods/b418109a-7d5d-445d-b6e1-312f4b2a4ad6/volumes"
Jul 23 12:11:22.438379 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:22.438348    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:11:26.391376 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:26.391348    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:11:26.398242 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:26.398207    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:11:26.408054 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:26.408035    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:11:26.414217 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:26.414198    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v2v4p_d1793efe-de3b-4383-a449-22ac51434713/ovn-acl-logging/0.log"
Jul 23 12:11:41.727047 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:41.727013    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:11:41.727509 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:41.727217    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" podUID="3d3764de-a179-403c-9474-d436ef99ae01" containerName="maas-api" containerID="cri-o://22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e" gracePeriod=30
Jul 23 12:11:41.977362 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:41.977296    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:11:42.097797 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.097766    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zlvp\" (UniqueName: \"kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp\") pod \"3d3764de-a179-403c-9474-d436ef99ae01\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") "
Jul 23 12:11:42.098010 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.097842    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") pod \"3d3764de-a179-403c-9474-d436ef99ae01\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") "
Jul 23 12:11:42.098010 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.097868    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca\") pod \"3d3764de-a179-403c-9474-d436ef99ae01\" (UID: \"3d3764de-a179-403c-9474-d436ef99ae01\") "
Jul 23 12:11:42.098383 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.098355    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d3764de-a179-403c-9474-d436ef99ae01" (UID: "3d3764de-a179-403c-9474-d436ef99ae01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 12:11:42.100084 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.100060    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d3764de-a179-403c-9474-d436ef99ae01" (UID: "3d3764de-a179-403c-9474-d436ef99ae01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 12:11:42.100253 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.100231    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp" (OuterVolumeSpecName: "kube-api-access-2zlvp") pod "3d3764de-a179-403c-9474-d436ef99ae01" (UID: "3d3764de-a179-403c-9474-d436ef99ae01"). InnerVolumeSpecName "kube-api-access-2zlvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 12:11:42.205896 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.205851    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zlvp\" (UniqueName: \"kubernetes.io/projected/3d3764de-a179-403c-9474-d436ef99ae01-kube-api-access-2zlvp\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:42.205896 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.205892    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d3764de-a179-403c-9474-d436ef99ae01-maas-api-tls\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:42.205896 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.205905    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d3764de-a179-403c-9474-d436ef99ae01-openshift-service-ca\") on node \"ip-10-0-140-183.ec2.internal\" DevicePath \"\""
Jul 23 12:11:42.313652 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.313612    2574 generic.go:356] "Generic (PLEG): container finished" podID="3d3764de-a179-403c-9474-d436ef99ae01" containerID="22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e" exitCode=0
Jul 23 12:11:42.313827 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.313674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" event={"ID":"3d3764de-a179-403c-9474-d436ef99ae01","Type":"ContainerDied","Data":"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"}
Jul 23 12:11:42.313827 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.313687    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"
Jul 23 12:11:42.313827 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.313716    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq" event={"ID":"3d3764de-a179-403c-9474-d436ef99ae01","Type":"ContainerDied","Data":"266c812f9be6c5243e52e1b0bae340e9308534d3e84d2800db6331e5e068b5e8"}
Jul 23 12:11:42.313827 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.313737    2574 scope.go:117] "RemoveContainer" containerID="22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"
Jul 23 12:11:42.322806 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.322787    2574 scope.go:117] "RemoveContainer" containerID="22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"
Jul 23 12:11:42.323063 ip-10-0-140-183 kubenswrapper[2574]: E0723 12:11:42.323044    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e\": container with ID starting with 22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e not found: ID does not exist" containerID="22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"
Jul 23 12:11:42.323122 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.323072    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e"} err="failed to get container status \"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e\": rpc error: code = NotFound desc = could not find container \"22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e\": container with ID starting with 22fe495a73cd2a8bb732881ba0421cdf50e139748281b3fc91293c5aaeb9aa1e not found: ID does not exist"
Jul 23 12:11:42.332401 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.332377    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:11:42.333994 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:42.333972    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-77f87c-758649fb94-ghtfq"]
Jul 23 12:11:44.239982 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:44.239948    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tncjv"]
Jul 23 12:11:44.297344 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:11:44.297314    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d3764de-a179-403c-9474-d436ef99ae01" path="/var/lib/kubelet/pods/3d3764de-a179-403c-9474-d436ef99ae01/volumes"
Jul 23 12:12:00.146017 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:00.145990    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674bfcc66c-z5r7z_90a30124-5518-4f73-8520-95aeb0d75a20/copy-manifests/0.log"
Jul 23 12:12:00.161034 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:00.161005    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674bfcc66c-z5r7z_90a30124-5518-4f73-8520-95aeb0d75a20/manager/0.log"
Jul 23 12:12:00.523243 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:00.523164    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6779586b89-d9pbz_dff04c43-1c40-4272-97f6-61adc3ca3b9e/maas-api/0.log"
Jul 23 12:12:02.920246 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:02.920214    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tncjv_617c5e2a-6ffb-46c8-963a-4cf8d39450cd/limitador/0.log"
Jul 23 12:12:03.401778 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:03.401752    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccd5f8bl_539eaa16-ce0d-4526-9fcf-e08556ca3f42/istio-proxy/0.log"
Jul 23 12:12:03.880882 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:03.880855    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/1.log"
Jul 23 12:12:04.255126 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:04.255052    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54c8b47ff6-g6ndr_702a7933-1e2c-4e40-9bc5-f0ed5ee0e139/router/0.log"
Jul 23 12:12:04.629858 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:04.629828    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-x9wbn_ce15ab8d-f2ca-4d43-9542-5c22af07a931/main/0.log"
Jul 23 12:12:05.103273 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:05.103242    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ls6w5_6f0cee02-745f-47b3-90d9-4b635b0b9ade/main/0.log"
Jul 23 12:12:05.215398 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:05.215369    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gwb86_403350ec-fc63-4224-ad1a-86a80749dc58/main/0.log"
Jul 23 12:12:13.764149 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:13.764116    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wzbq5_14e709f1-29c4-4514-ac9f-35b04391b96a/global-pull-secret-syncer/0.log"
Jul 23 12:12:13.898854 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:13.898804    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n9f5t_efcb7ca5-ada5-4b47-9a30-1591948a7f0e/konnectivity-agent/0.log"
Jul 23 12:12:13.977219 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:13.977186    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-183.ec2.internal_50bdc59f0c097001cebef932b3a8af10/haproxy/0.log"
Jul 23 12:12:19.311115 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:19.311077    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tncjv_617c5e2a-6ffb-46c8-963a-4cf8d39450cd/limitador/0.log"
Jul 23 12:12:21.469400 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:21.469365    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-dgxv2_9d4e3a23-07a2-4660-9ec8-a1a43324d0f0/cluster-monitoring-operator/0.log"
Jul 23 12:12:21.661370 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:21.661324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27dxq_2c81da6d-2852-4357-b28c-adfa804e7ad3/node-exporter/0.log"
Jul 23 12:12:21.690060 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:21.689985    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27dxq_2c81da6d-2852-4357-b28c-adfa804e7ad3/kube-rbac-proxy/0.log"
Jul 23 12:12:21.723897 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:21.723867    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-27dxq_2c81da6d-2852-4357-b28c-adfa804e7ad3/init-textfile/0.log"
Jul 23 12:12:23.758627 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.758593    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-xnm87_0d7112e7-a9f5-4bb3-b7cf-6fe9e6c0e47a/networking-console-plugin/0.log"
Jul 23 12:12:23.841820 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.841788    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"]
Jul 23 12:12:23.842229 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842214    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d3764de-a179-403c-9474-d436ef99ae01" containerName="maas-api"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842231    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d3764de-a179-403c-9474-d436ef99ae01" containerName="maas-api"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842239    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c2b196b-93f2-43f6-845b-0a74701e91ed" containerName="main"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842245    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c2b196b-93f2-43f6-845b-0a74701e91ed" containerName="main"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842253    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" containerName="main"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842258    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" containerName="main"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842265    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" containerName="maas-api"
Jul 23 12:12:23.842275 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842270    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" containerName="maas-api"
Jul 23 12:12:23.842518 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842330    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a2e4ae8-89b2-4dca-9a94-5719e94b8f94" containerName="main"
Jul 23 12:12:23.842518 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842340    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b418109a-7d5d-445d-b6e1-312f4b2a4ad6" containerName="maas-api"
Jul 23 12:12:23.842518 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842348    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c2b196b-93f2-43f6-845b-0a74701e91ed" containerName="main"
Jul 23 12:12:23.842518 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.842354    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d3764de-a179-403c-9474-d436ef99ae01" containerName="maas-api"
Jul 23 12:12:23.845793 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.845771    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.848534 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.848511    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xw2jd\"/\"openshift-service-ca.crt\""
Jul 23 12:12:23.849499 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.849477    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xw2jd\"/\"default-dockercfg-k7f76\""
Jul 23 12:12:23.849499 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.849495    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xw2jd\"/\"kube-root-ca.crt\""
Jul 23 12:12:23.857888 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.857863    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"]
Jul 23 12:12:23.858263 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.858242    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-proc\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.858337 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.858291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-lib-modules\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.858392 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.858357    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-podres\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.858392 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.858378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-sys\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.858509 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.858395    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7x7n\" (UniqueName: \"kubernetes.io/projected/bf015b00-a920-47c6-a09b-1b4212868604-kube-api-access-w7x7n\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.958862 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-lib-modules\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-podres\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958906    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-sys\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958931    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7x7n\" (UniqueName: \"kubernetes.io/projected/bf015b00-a920-47c6-a09b-1b4212868604-kube-api-access-w7x7n\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958989    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-sys\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.958995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-proc\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.959019    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-lib-modules\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.959034    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-proc\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.959059 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.959021    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bf015b00-a920-47c6-a09b-1b4212868604-podres\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:23.971093 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:23.971065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7x7n\" (UniqueName: \"kubernetes.io/projected/bf015b00-a920-47c6-a09b-1b4212868604-kube-api-access-w7x7n\") pod \"perf-node-gather-daemonset-c9nxw\" (UID: \"bf015b00-a920-47c6-a09b-1b4212868604\") " pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:24.156750 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.156712    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:24.301269 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.301243    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"]
Jul 23 12:12:24.303559 ip-10-0-140-183 kubenswrapper[2574]: W0723 12:12:24.303522    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbf015b00_a920_47c6_a09b_1b4212868604.slice/crio-54a776e9719377e012a9701bfeb2ec0ad2a3f737ac5c981ee5826ff4ea3c0417 WatchSource:0}: Error finding container 54a776e9719377e012a9701bfeb2ec0ad2a3f737ac5c981ee5826ff4ea3c0417: Status 404 returned error can't find the container with id 54a776e9719377e012a9701bfeb2ec0ad2a3f737ac5c981ee5826ff4ea3c0417
Jul 23 12:12:24.500769 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.500686    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/1.log"
Jul 23 12:12:24.505671 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.505636    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-4dp7h_a6a41630-31a9-4f8d-a321-be18df3b14e9/console-operator/2.log"
Jul 23 12:12:24.512101 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.512065    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw" event={"ID":"bf015b00-a920-47c6-a09b-1b4212868604","Type":"ContainerStarted","Data":"8cfda5c6dd4ebb3276ebfa725d78898c0dff72ba8943183a9ea582704e009d2b"}
Jul 23 12:12:24.512224 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.512138    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw" event={"ID":"bf015b00-a920-47c6-a09b-1b4212868604","Type":"ContainerStarted","Data":"54a776e9719377e012a9701bfeb2ec0ad2a3f737ac5c981ee5826ff4ea3c0417"}
Jul 23 12:12:24.512270 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.512215    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:24.535450 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:24.535397    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw" podStartSLOduration=1.5353815389999999 podStartE2EDuration="1.535381539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:12:24.533965745 +0000 UTC m=+2758.819431501" watchObservedRunningTime="2026-07-23 12:12:24.535381539 +0000 UTC m=+2758.820847335"
Jul 23 12:12:25.141035 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:25.140992    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-fz5kc_41641bf8-7229-46c8-9881-6b111a01ff21/download-server/0.log"
Jul 23 12:12:25.745457 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:25.745423    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-njk2f_8781f6f3-451a-4393-a575-e16ac00e2687/volume-data-source-validator/0.log"
Jul 23 12:12:27.031353 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:27.031323    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vjptn_5992ce67-f59b-4fbd-ab61-0708b33410df/dns/0.log"
Jul 23 12:12:27.073547 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:27.073519    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vjptn_5992ce67-f59b-4fbd-ab61-0708b33410df/kube-rbac-proxy/0.log"
Jul 23 12:12:27.166900 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:27.166873    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kbssr_9f2feb0a-1fb1-4137-abcb-64b3df097c96/dns-node-resolver/0.log"
Jul 23 12:12:27.816573 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:27.816542    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pg9xf_64cf79fc-0c55-4ab7-8d1f-b41c47e5cc25/node-ca/0.log"
Jul 23 12:12:28.845532 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:28.845500    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccd5f8bl_539eaa16-ce0d-4526-9fcf-e08556ca3f42/istio-proxy/0.log"
Jul 23 12:12:29.062984 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:29.062951    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/0.log"
Jul 23 12:12:29.091006 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:29.090979    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/1.log"
Jul 23 12:12:29.186527 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:29.186416    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54c8b47ff6-g6ndr_702a7933-1e2c-4e40-9bc5-f0ed5ee0e139/router/0.log"
Jul 23 12:12:29.934213 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:29.934182    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5c96j_5d62d78d-4f12-4db9-948f-f809d037f32f/serve-healthcheck-canary/0.log"
Jul 23 12:12:30.515403 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.515372    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-g295d_90b4c78f-dfd7-4365-8510-c0bf3a7acac8/insights-operator/0.log"
Jul 23 12:12:30.515963 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.515945    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-g295d_90b4c78f-dfd7-4365-8510-c0bf3a7acac8/insights-operator/1.log"
Jul 23 12:12:30.528633 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.528607    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xw2jd/perf-node-gather-daemonset-c9nxw"
Jul 23 12:12:30.729441 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.729412    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4hbv_1adbad02-1655-44ea-b057-5629f0df8c47/kube-rbac-proxy/0.log"
Jul 23 12:12:30.759550 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.759517    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4hbv_1adbad02-1655-44ea-b057-5629f0df8c47/exporter/0.log"
Jul 23 12:12:30.791406 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:30.791332    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q4hbv_1adbad02-1655-44ea-b057-5629f0df8c47/extractor/0.log"
Jul 23 12:12:34.059917 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.059880    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674bfcc66c-z5r7z_90a30124-5518-4f73-8520-95aeb0d75a20/manager/0.log"
Jul 23 12:12:34.080003 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.079974    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-674bfcc66c-z5r7z_90a30124-5518-4f73-8520-95aeb0d75a20/copy-manifests/0.log"
Jul 23 12:12:34.390261 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.390232    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccd5f8bl_539eaa16-ce0d-4526-9fcf-e08556ca3f42/istio-proxy/0.log"
Jul 23 12:12:34.568909 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.568878    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/0.log"
Jul 23 12:12:34.598041 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.598003    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/1.log"
Jul 23 12:12:34.662226 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:34.662147    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54c8b47ff6-g6ndr_702a7933-1e2c-4e40-9bc5-f0ed5ee0e139/router/0.log"
Jul 23 12:12:35.355179 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:35.355151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c45f5478c-t5g8z_82e0d4cc-a02f-4faa-838b-66bb439ff7dd/manager/0.log"
Jul 23 12:12:35.875761 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:35.875733    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-656c84dccd5f8bl_539eaa16-ce0d-4526-9fcf-e08556ca3f42/istio-proxy/0.log"
Jul 23 12:12:36.062510 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:36.062419    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/0.log"
Jul 23 12:12:36.088228 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:36.088181    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5d7b8c9489-6zqb4_35d4ad14-eb9c-4ded-8688-066b4681eed8/istio-proxy/1.log"
Jul 23 12:12:36.150434 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:36.150405    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54c8b47ff6-g6ndr_702a7933-1e2c-4e40-9bc5-f0ed5ee0e139/router/0.log"
Jul 23 12:12:40.895795 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:40.895764    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-85b2p_48f361ec-0e22-4640-820f-0eb38d1b956c/migrator/0.log"
Jul 23 12:12:40.926858 ip-10-0-140-183 kubenswrapper[2574]: I0723 12:12:40.926833    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-85b2p_48f361ec-0e22-4640-820f-0eb38d1b956c/graceful-termination/0.log"