Jul 17 14:49:49.310318 ip-10-0-138-81 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 14:49:49.721222 ip-10-0-138-81 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 14:49:49.721222 ip-10-0-138-81 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 14:49:49.721222 ip-10-0-138-81 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 14:49:49.721222 ip-10-0-138-81 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 14:49:49.721222 ip-10-0-138-81 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 14:49:49.724312 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.724204    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 14:49:49.732249 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732228    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.732249 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732245    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.732249 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732249    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.732249 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732252    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.732249 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732255    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732260    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732265    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732269    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732272    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732274    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732277    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732280    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732282    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732285    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732287    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732290    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732295    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732297    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732300    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732302    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732305    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732307    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732310    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732312    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.732453 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732315    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732321    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732324    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732327    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732329    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732332    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732334    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732337    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732340    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732342    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732345    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732347    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732350    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732352    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732356    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732361    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732365    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732368    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732371    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732374    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.732925 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732377    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732380    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732383    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732385    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732388    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732391    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732393    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732396    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732399    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732402    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732404    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732407    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732409    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732411    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732414    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732416    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732419    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732421    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732424    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732426    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732429    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.733444 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732431    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732435    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732439    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732442    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732445    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732448    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732452    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732455    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732458    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732460    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732463    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732466    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732469    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732471    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732474    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732476    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732480    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732485    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732488    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.733929 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732491    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732494    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732498    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732501    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732503    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732506    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732508    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732511    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732513    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732516    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732518    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732521    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732524    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732526    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.732528    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733233    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733239    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733242    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733245    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.734404 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733248    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733251    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733254    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733256    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733259    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733261    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733264    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733267    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733269    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733273    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733277    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733280    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733282    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733286    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733288    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733291    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733293    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733297    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733299    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733302    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.734900 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733304    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733309    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733312    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733316    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733318    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733321    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733324    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733326    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733329    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733332    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733334    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733337    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733340    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733343    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733345    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733348    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733351    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733354    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733356    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.735439 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733359    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733362    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733364    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733367    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733370    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733373    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733375    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733378    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733381    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733384    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733387    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733389    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733392    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733394    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733397    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733400    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733403    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733405    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733408    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733410    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.735888 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733413    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733415    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733418    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733420    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733423    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733426    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733428    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733431    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733433    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733436    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733438    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733442    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733444    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733447    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733449    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733452    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733455    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733458    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733460    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733463    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.736382 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733479    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733483    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733485    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733488    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733491    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733494    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733496    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733499    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733501    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733504    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733506    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733509    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733511    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733514    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733517    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733519    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733945    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733951    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733954    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.736858 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733958    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733962    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733965    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733968    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733971    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733974    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733976    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733979    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733981    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733984    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733986    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733989    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733992    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733996    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.733998    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734001    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734003    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734006    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734008    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734011    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.737323 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734014    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734016    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734019    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734021    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734024    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734026    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734029    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734031    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734033    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734036    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734039    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734042    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734044    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734047    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734050    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734052    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734055    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734057    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734060    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734080    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.737806 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734084    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734087    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734091    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734095    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734098    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734101    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734106    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734109    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734112    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734115    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734117    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734120    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734122    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734126    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734129    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734132    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734134    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734136    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734140    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.738298 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734143    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734146    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734152    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734154    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734157    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734160    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734162    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734165    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734167    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734170    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734173    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734175    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734178    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734180    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734183    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734185    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734188    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734190    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734193    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734196    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.738758 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734199    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734203    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734206    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734209    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734211    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734214    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734216    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734219    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734221    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734224    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734227    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734229    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734231    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734234    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734238    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734240    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734243    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734311    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734318    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734324    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734328    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 14:49:49.739259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734332    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734335    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734340    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734343    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734347    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734350    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734353    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734357    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734359    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734362    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734365    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734368    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734372    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734375    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734379    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734382    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734385    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734388    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734391    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734394    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734397    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734400    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734403    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734406    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 14:49:49.739756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734409    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734414    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734418    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734421    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734427    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734430    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734433    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734436    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734439    2564 flags.go:64] FLAG: --enable-server="true"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734446    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734450    2564 flags.go:64] FLAG: --event-burst="100"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734453    2564 flags.go:64] FLAG: --event-qps="50"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734456    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734459    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734462    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734465    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734468    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734471    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734474    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734476    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734479    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734483    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734486    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734489    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734492    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 14:49:49.740327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734495    2564 flags.go:64] FLAG: --feature-gates=""
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734498    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734501    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734504    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734507    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734510    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734513    2564 flags.go:64] FLAG: --help="false"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734516    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-138-81.ec2.internal"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734519    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734522    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734525    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734528    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734532    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734534    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734537    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734540    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734544    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734547    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734550    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734553    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734555    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734558    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734561    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734564    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 14:49:49.740985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734566    2564 flags.go:64] FLAG: --lock-file=""
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734578    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734582    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734585    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734590    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734594    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734597    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734600    2564 flags.go:64] FLAG: --logging-format="text"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734602    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734606    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734608    2564 flags.go:64] FLAG: --manifest-url=""
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734611    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734615    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734618    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734621    2564 flags.go:64] FLAG: --max-pods="110"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734624    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734627    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734630    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734633    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734636    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734639    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734642    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734649    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734652    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734656    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 14:49:49.741577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734659    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734662    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734667    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734670    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734673    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734676    2564 flags.go:64] FLAG: --port="10250"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734679    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734682    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00fa07ac69c80cf77"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734685    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734688    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734690    2564 flags.go:64] FLAG: --register-node="true"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734693    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734696    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734700    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734703    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734705    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734709    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734712    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734715    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734717    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734720    2564 flags.go:64] FLAG: --runonce="false"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734723    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734726    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734729    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734731    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 14:49:49.742191 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734734    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734737    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734740    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734743    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734745    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734748    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734752    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734755    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734758    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734761    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734763    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734768    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734771    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734774    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734781    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734784    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734786    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734789    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734792    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734795    2564 flags.go:64] FLAG: --v="2"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734799    2564 flags.go:64] FLAG: --version="false"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734803    2564 flags.go:64] FLAG: --vmodule=""
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734806    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.734809    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734920    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.742797 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734924    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734927    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734930    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734932    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734935    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734938    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734941    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734944    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734947    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734949    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734952    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734954    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734957    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734961    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734963    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734966    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734968    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734970    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734973    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734976    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.743396 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734978    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734981    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734983    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734985    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734988    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734991    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734993    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734996    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.734998    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735001    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735005    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735009    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735011    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735014    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735017    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735020    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735022    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735025    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735029    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735031    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.743966 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735034    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735036    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735039    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735041    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735044    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735047    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735050    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735052    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735055    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735057    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735060    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735075    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735078    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735081    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735083    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735086    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735088    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735091    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735093    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735096    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.744484 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735098    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735101    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735104    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735106    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735109    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735111    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735114    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735116    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735120    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735124    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735129    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735132    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735134    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735137    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735154    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735158    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735161    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735165    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735168    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.744961 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735171    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735174    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735178    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735181    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735184    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735187    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735190    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735192    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735195    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735198    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735201    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735204    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735206    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735209    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735211    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735214    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735217    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735219    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.745435 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.735222    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.735824    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.742831    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.742850    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.742871    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.742877    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742935    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742940    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742944    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742947    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742950    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742959    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742962    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742965    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742968    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.745863 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742970    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742973    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742976    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742979    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742982    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742984    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742987    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742989    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742992    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742994    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742997    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.742999    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743002    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743004    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743007    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743009    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743012    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743015    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743017    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743020    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.746235 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743029    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743032    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743035    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743037    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743040    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743043    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743045    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743048    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743050    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743053    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743056    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743058    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743075    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743079    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743081    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743084    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743087    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743089    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743092    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743095    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.746723 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743097    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743100    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743103    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743105    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743108    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743111    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743114    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743116    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743119    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743121    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743124    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743127    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743130    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743138    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743141    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743144    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743148    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743151    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743154    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.747262 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743157    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743160    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743163    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743166    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743168    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743171    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743173    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743176    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743178    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743181    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743183    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743186    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743188    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743191    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743194    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743196    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743199    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743201    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743204    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.747701 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743207    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743209    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743213    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743216    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743220    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743224    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743227    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743230    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743240    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743243    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743246    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743248    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.743253    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743392    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 14:49:49.748172 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743395    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743398    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743401    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743404    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743407    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743409    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743412    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743414    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743417    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743419    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743423    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743426    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743429    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743432    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743434    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743437    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743439    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743442    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743444    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 14:49:49.748512 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743446    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743449    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743451    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743454    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743456    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743459    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743461    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743464    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743473    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743476    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743479    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743481    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743484    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743486    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743489    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743491    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743493    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743497    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743500    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 14:49:49.748956 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743503    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743506    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743509    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743511    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743514    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743516    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743519    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743521    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743524    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743527    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743530    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743532    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743534    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743537    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743539    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743542    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743545    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743547    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743549    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743552    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 14:49:49.749413 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743554    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743557    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743565    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743568    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743570    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743573    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743575    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743578    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743580    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743582    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743585    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743587    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743589    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743592    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743594    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743597    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743599    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743602    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743604    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743606    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 14:49:49.749884 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743609    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743611    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743613    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743616    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743618    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743621    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743623    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743626    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743628    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743632    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743635    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743639    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743642    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743644    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743647    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743655    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743658    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743661    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743663    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 14:49:49.750359 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:49.743665    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.743670    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.743772    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.747781    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.749291    2564 server.go:1019] "Starting client certificate rotation"
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.749392    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 14:49:49.750815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.749424    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 14:49:49.751843 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.751830    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 14:49:49.753252 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.753237    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 14:49:49.767207 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.767187    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 17 14:49:49.770363 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.770350    2564 log.go:25] "Validated CRI v1 image API"
Jul 17 14:49:49.771174 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.771161    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 14:49:49.773696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.773675    2564 fs.go:135] Filesystem UUIDs: map[07795c9e-9737-4d01-b5ce-d20fcd7815ef:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9e95fbaf-3873-4661-a5ff-1761920cb3c2:/dev/nvme0n1p3]
Jul 17 14:49:49.773774 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.773695    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 14:49:49.778981 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.778868    2564 manager.go:216] Machine: {Timestamp:2026-07-17 14:49:49.778396483 +0000 UTC m=+0.345427607 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097212 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c44c31c07825e6de714d0fde53365 SystemUUID:ec2c44c3-1c07-825e-6de7-14d0fde53365 BootID:0bf4aa32-fe64-43f6-a162-962806ce6a61 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:e8:6d:23:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:e8:6d:23:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:09:27:61:22:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 14:49:49.778981 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.778967    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 14:49:49.779150 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.779058    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 14:49:49.781645 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.781622    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 14:49:49.781768 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.781646    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 14:49:49.782434 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.782423    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 14:49:49.782465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.782436    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 14:49:49.782465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.782443    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 14:49:49.783382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.783371    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 14:49:49.784674 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.784664    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 14:49:49.788578 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788565    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 14:49:49.788687 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788678    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 14:49:49.788724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788706    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 14:49:49.788724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788715    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 14:49:49.788805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788728    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 14:49:49.788805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788737    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 17 14:49:49.788805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.788745    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 14:49:49.789079 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.789049    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 14:49:49.789182 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.789091    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 14:49:49.790656 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.790638    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 14:49:49.791282 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.791265    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 14:49:49.793285 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.793272    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 14:49:49.793341 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.793302    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 14:49:49.794792 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794780    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794796    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794803    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794810    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794818    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794823    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794831    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794836    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794842    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 14:49:49.794844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794847    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 14:49:49.795095 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794861    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 14:49:49.795095 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.794866    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 14:49:49.796192 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.796179    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 14:49:49.796192 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.796191    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 14:49:49.799149 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.799136    2564 server.go:1268] "Started kubelet"
Jul 17 14:49:49.799264 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.799213    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 14:49:49.799264 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.799225    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 14:49:49.799364 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.799266    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 14:49:49.799877 ip-10-0-138-81 systemd[1]: Started Kubernetes Kubelet.
Jul 17 14:49:49.801324 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.801304    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 14:49:49.801409 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.801393    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 14:49:49.805738 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.805718    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 14:49:49.805862 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.805787    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 14:49:49.805986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.805950    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 14:49:49.806316 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.806300    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 14:49:49.810561 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.809474    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-81.ec2.internal.18c31b3487c7c235  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-81.ec2.internal,UID:ip-10-0-138-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-81.ec2.internal,},FirstTimestamp:2026-07-17 14:49:49.799113269 +0000 UTC m=+0.366144392,LastTimestamp:2026-07-17 14:49:49.799113269 +0000 UTC m=+0.366144392,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-81.ec2.internal,}"
Jul 17 14:49:49.813116 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813091    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 14:49:49.813213 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.813189    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:49.813213 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813093    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 14:49:49.813920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813900    2564 factory.go:55] Registering systemd factory
Jul 17 14:49:49.813920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813922    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 17 14:49:49.814054 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813935    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 14:49:49.814054 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813938    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 14:49:49.814054 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.813959    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 14:49:49.814209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814078    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 14:49:49.814209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814086    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 14:49:49.814370 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814351    2564 factory.go:153] Registering CRI-O factory
Jul 17 14:49:49.814426 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814372    2564 factory.go:223] Registration of the crio container factory successfully
Jul 17 14:49:49.814461 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814447    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 14:49:49.814510 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814501    2564 factory.go:103] Registering Raw factory
Jul 17 14:49:49.814559 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814516    2564 manager.go:1196] Started watching for new ooms in manager
Jul 17 14:49:49.814914 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.814902    2564 manager.go:349] Starting recovery of all containers
Jul 17 14:49:49.820877 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.820818    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 14:49:49.820877 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.820861    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 14:49:49.824211 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.824197    2564 manager.go:354] Recovery completed
Jul 17 14:49:49.828312 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.828300    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:49.830623 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.830588    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:49.830702 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.830631    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:49.830702 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.830645    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:49.831825 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.831807    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 14:49:49.831825 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.831825    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 14:49:49.831949 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.831816    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:49.831949 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.831846    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 14:49:49.832552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.832539    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:49.832627 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.832565    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:49.832627 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.832574    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:49.833457 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.833444    2564 policy_none.go:49] "None policy: Start"
Jul 17 14:49:49.833511 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.833460    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 14:49:49.833511 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.833470    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 14:49:49.834975 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.834964    2564 policy_none.go:47] "Start"
Jul 17 14:49:49.837103 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.837047    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r87bg"
Jul 17 14:49:49.845766 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.845692    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-81.ec2.internal.18c31b3489a86f93  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-81.ec2.internal,UID:ip-10-0-138-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-81.ec2.internal,},FirstTimestamp:2026-07-17 14:49:49.830614931 +0000 UTC m=+0.397646060,LastTimestamp:2026-07-17 14:49:49.830614931 +0000 UTC m=+0.397646060,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-81.ec2.internal,}"
Jul 17 14:49:49.851398 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.851382    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r87bg"
Jul 17 14:49:49.859430 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.859201    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-81.ec2.internal.18c31b3489a8c703  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-81.ec2.internal,UID:ip-10-0-138-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-81.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-81.ec2.internal,},FirstTimestamp:2026-07-17 14:49:49.830637315 +0000 UTC m=+0.397668439,LastTimestamp:2026-07-17 14:49:49.830637315 +0000 UTC m=+0.397668439,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-81.ec2.internal,}"
Jul 17 14:49:49.869634 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.869616    2564 manager.go:337] "Starting Device Plugin manager"
Jul 17 14:49:49.869720 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.869654    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 14:49:49.869720 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.869664    2564 server.go:87] "Starting device plugin registration server"
Jul 17 14:49:49.869906 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.869894    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 14:49:49.869943 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.869908    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 14:49:49.870018 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.869990    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 14:49:49.870110 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.870098    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 14:49:49.870155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.870112    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 14:49:49.870605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.870586    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 14:49:49.870672 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.870637    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:49.942526 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.942494    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 14:49:49.943902 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.943597    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 14:49:49.943902 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.943629    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 14:49:49.943902 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.943651    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 14:49:49.943902 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.943682    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 14:49:49.948585 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.948558    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 14:49:49.970187 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.970172    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:49.971018 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.971000    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:49.971122 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.971033    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:49.971122 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.971046    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:49.971122 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.971086    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-81.ec2.internal"
Jul 17 14:49:49.985630 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:49.985583    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-81.ec2.internal"
Jul 17 14:49:49.985630 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:49.985604    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-81.ec2.internal\": node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.011514 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.011492    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.044656 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.044633    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"]
Jul 17 14:49:50.044747 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.044698    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:50.046307 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.046288    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:50.046425 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.046320    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:50.046425 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.046330    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:50.047935 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.047923    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:50.048034 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.048088 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048051    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:50.048659 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048640    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:50.048755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048674    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:50.048755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048689    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:50.048755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048640    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:50.048755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048737    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:50.048755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.048750    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:50.050064 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.050050    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.050163 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.050090    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:50.050718 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.050703    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:50.050780 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.050729    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:50.050780 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.050744    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:50.075666 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.075650    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-81.ec2.internal\" not found" node="ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.080073 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.080057    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-81.ec2.internal\" not found" node="ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.112203 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.112183    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.115565 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.115549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5708bb1a977bc61e2607d3ed9ee9c042-config\") pod \"kube-apiserver-proxy-ip-10-0-138-81.ec2.internal\" (UID: \"5708bb1a977bc61e2607d3ed9ee9c042\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.115614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.115579    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.115614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.115600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.213107 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.213084    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.216435 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5708bb1a977bc61e2607d3ed9ee9c042-config\") pod \"kube-apiserver-proxy-ip-10-0-138-81.ec2.internal\" (UID: \"5708bb1a977bc61e2607d3ed9ee9c042\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.216482 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216439    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5708bb1a977bc61e2607d3ed9ee9c042-config\") pod \"kube-apiserver-proxy-ip-10-0-138-81.ec2.internal\" (UID: \"5708bb1a977bc61e2607d3ed9ee9c042\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.216482 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.216482 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.216577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216517    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.216577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.216550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a178bbce5cd9e6ffa79c6b7438d62c16-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal\" (UID: \"a178bbce5cd9e6ffa79c6b7438d62c16\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.313249 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.313181    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.379204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.379171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.382437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.382418    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:50.413829 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.413806    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.514691 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.514647    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.615373 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.615304    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.716017 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.715984    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.748396 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.748374    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 14:49:50.748796 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.748536    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 14:49:50.816116 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.816078    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.858614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.858577    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 14:44:49 +0000 UTC" deadline="2027-12-16 03:20:30.983327324 +0000 UTC"
Jul 17 14:49:50.858614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.858605    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12396h30m40.124726145s"
Jul 17 14:49:50.916525 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:50.916502    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:50.921713 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.921702    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 14:49:50.922982 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.922964    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 14:49:50.923076 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.922994    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 14:49:50.923076 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.923004    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeHasSufficientPID"
Jul 17 14:49:50.991168 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:50.991141    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5708bb1a977bc61e2607d3ed9ee9c042.slice/crio-3ab32da62b5c219e19af0e7186f8a94aeff0475065b0cddad11ce4190c2ff81c WatchSource:0}: Error finding container 3ab32da62b5c219e19af0e7186f8a94aeff0475065b0cddad11ce4190c2ff81c: Status 404 returned error can't find the container with id 3ab32da62b5c219e19af0e7186f8a94aeff0475065b0cddad11ce4190c2ff81c
Jul 17 14:49:50.991653 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:50.991635    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda178bbce5cd9e6ffa79c6b7438d62c16.slice/crio-d4608b982183970dca14409f5a7f08f786900df56e090e2ec4cf3bb9c31b63a2 WatchSource:0}: Error finding container d4608b982183970dca14409f5a7f08f786900df56e090e2ec4cf3bb9c31b63a2: Status 404 returned error can't find the container with id d4608b982183970dca14409f5a7f08f786900df56e090e2ec4cf3bb9c31b63a2
Jul 17 14:49:50.996584 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:50.996568    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 14:49:51.001303 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.001282    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 14:49:51.017048 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:51.017023    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:51.117519 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:51.117496    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-81.ec2.internal\" not found"
Jul 17 14:49:51.158411 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.158346    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 14:49:51.214370 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.214338    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:51.244423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.244394    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 14:49:51.245634 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.245619    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"
Jul 17 14:49:51.268952 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.268934    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 14:49:51.274364 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.274346    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 14:49:51.790989 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.790961    2564 apiserver.go:52] "Watching apiserver"
Jul 17 14:49:51.796109 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.796085    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 14:49:51.796390 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.796366    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6hdkd","kube-system/konnectivity-agent-mtglb","kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n","openshift-cluster-node-tuning-operator/tuned-qzw8g","openshift-image-registry/node-ca-qmhfb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal"]
Jul 17 14:49:51.798210 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.798190    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.798307 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:51.798271    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:49:51.800431 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.800408    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.802477 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.802454    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.804044 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.804025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.805615 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.805597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.814142 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.814125    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 14:49:51.814381 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.814365    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 14:49:51.823873 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.823855    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 14:49:51.823981 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.823855    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 14:49:51.824192 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824175    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 14:49:51.824238 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824220    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 14:49:51.824304 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824283    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 14:49:51.824355 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824315    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 14:49:51.824403 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824361    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:49:51.824453 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysconfig\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824505 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-konnectivity-ca\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.824505 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-registration-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.824603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824513    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.824603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824538    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-lib-modules\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-host\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-dbus\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.824705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93e485cf-83b7-43d4-99bd-d9882c233d5b-serviceca\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.824705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824656    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.824705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824679    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57fnj\" (UniqueName: \"kubernetes.io/projected/0054df8e-6154-4982-8267-2c92e3895772-kube-api-access-57fnj\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-agent-certs\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-systemd\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-sys\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-etc-tuned\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-tmp\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824825    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-kubelet-config\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824837    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-clq4r\""
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824849    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-sys-fs\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.824876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-run\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824895    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-modprobe-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824918    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmdrm\" (UniqueName: \"kubernetes.io/projected/93e485cf-83b7-43d4-99bd-d9882c233d5b-kube-api-access-rmdrm\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-device-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmp4b\" (UniqueName: \"kubernetes.io/projected/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kube-api-access-mmp4b\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.824995    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-kubernetes\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-var-lib-kubelet\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-socket-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-conf\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825125    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.825184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.825140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93e485cf-83b7-43d4-99bd-d9882c233d5b-host\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.828011 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.827991    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vhzfk\""
Jul 17 14:49:51.828140 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.828047    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 14:49:51.828140 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.828098    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 14:49:51.828288 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.828270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-58knt\""
Jul 17 14:49:51.828391 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.828371    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 14:49:51.828832 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.828819    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-85k9l\""
Jul 17 14:49:51.848791 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.848768    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 14:49:51.860380 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.860357    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nsddm"
Jul 17 14:49:51.876169 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.876148    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nsddm"
Jul 17 14:49:51.896225 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.896204    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 14:49:51.926003 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.925982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-kubelet-config\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.926117 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-sys-fs\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926117 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-run\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926117 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926055    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-modprobe-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926117 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926093    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdrm\" (UniqueName: \"kubernetes.io/projected/93e485cf-83b7-43d4-99bd-d9882c233d5b-kube-api-access-rmdrm\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.926249 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-device-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926249 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926131    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-kubelet-config\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.926249 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmp4b\" (UniqueName: \"kubernetes.io/projected/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kube-api-access-mmp4b\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926249 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-sys-fs\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-modprobe-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-kubernetes\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-var-lib-kubelet\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-socket-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-conf\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926383    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-run\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93e485cf-83b7-43d4-99bd-d9882c233d5b-host\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysconfig\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-device-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-konnectivity-ca\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926516    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-var-lib-kubelet\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-registration-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-lib-modules\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-host\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-dbus\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93e485cf-83b7-43d4-99bd-d9882c233d5b-serviceca\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-conf\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926652    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926659    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysconfig\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57fnj\" (UniqueName: \"kubernetes.io/projected/0054df8e-6154-4982-8267-2c92e3895772-kube-api-access-57fnj\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-agent-certs\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926701    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93e485cf-83b7-43d4-99bd-d9882c233d5b-host\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-systemd\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926744    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-sys\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-etc-tuned\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-tmp\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-socket-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926486    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-kubernetes\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.926921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926903    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-etc-selinux\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-lib-modules\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.926567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-sysctl-d\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-host\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927104    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927179    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3f0b694-edc9-4e75-81de-7b528fa2195f-dbus\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-konnectivity-ca\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-etc-systemd\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:51.927383    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:51.927436    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:49:52.42741863 +0000 UTC m=+2.994449745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:51.927536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0054df8e-6154-4982-8267-2c92e3895772-sys\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.927896 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93e485cf-83b7-43d4-99bd-d9882c233d5b-serviceca\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.927896 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.927710    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/58562432-e34f-4a93-b92a-0a73d3e0e3d1-registration-dir\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.930313 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.930287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-etc-tuned\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.930420 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.930314    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0054df8e-6154-4982-8267-2c92e3895772-tmp\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:51.930531 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.930514    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e79ace2d-20ee-49ab-8d5d-c465f24d0c76-agent-certs\") pod \"konnectivity-agent-mtglb\" (UID: \"e79ace2d-20ee-49ab-8d5d-c465f24d0c76\") " pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:51.947490 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.947471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdrm\" (UniqueName: \"kubernetes.io/projected/93e485cf-83b7-43d4-99bd-d9882c233d5b-kube-api-access-rmdrm\") pod \"node-ca-qmhfb\" (UID: \"93e485cf-83b7-43d4-99bd-d9882c233d5b\") " pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:51.948520 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.948455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal" event={"ID":"5708bb1a977bc61e2607d3ed9ee9c042","Type":"ContainerStarted","Data":"3ab32da62b5c219e19af0e7186f8a94aeff0475065b0cddad11ce4190c2ff81c"}
Jul 17 14:49:51.949596 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.949555    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmp4b\" (UniqueName: \"kubernetes.io/projected/58562432-e34f-4a93-b92a-0a73d3e0e3d1-kube-api-access-mmp4b\") pod \"aws-ebs-csi-driver-node-sz29n\" (UID: \"58562432-e34f-4a93-b92a-0a73d3e0e3d1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:51.949758 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.949739    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal" event={"ID":"a178bbce5cd9e6ffa79c6b7438d62c16","Type":"ContainerStarted","Data":"d4608b982183970dca14409f5a7f08f786900df56e090e2ec4cf3bb9c31b63a2"}
Jul 17 14:49:51.950829 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:51.950808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57fnj\" (UniqueName: \"kubernetes.io/projected/0054df8e-6154-4982-8267-2c92e3895772-kube-api-access-57fnj\") pod \"tuned-qzw8g\" (UID: \"0054df8e-6154-4982-8267-2c92e3895772\") " pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:52.111953 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.111850    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:52.117675 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.117651    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n"
Jul 17 14:49:52.124561 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.124540    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qzw8g"
Jul 17 14:49:52.128971 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.128952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qmhfb"
Jul 17 14:49:52.429624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.429523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:52.429795 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:52.429656    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:52.429795 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:52.429724    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:49:53.429704671 +0000 UTC m=+3.996735780 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:52.772284 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:52.772162    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79ace2d_20ee_49ab_8d5d_c465f24d0c76.slice/crio-41911e8389a854b4c2305220685d4f58991c6e48b66c0628f2ec1b7af884a4e8 WatchSource:0}: Error finding container 41911e8389a854b4c2305220685d4f58991c6e48b66c0628f2ec1b7af884a4e8: Status 404 returned error can't find the container with id 41911e8389a854b4c2305220685d4f58991c6e48b66c0628f2ec1b7af884a4e8
Jul 17 14:49:52.772440 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:52.772418    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e485cf_83b7_43d4_99bd_d9882c233d5b.slice/crio-11a9ba557cd56428a09c535912c678ba91074e2aaa9f3cd8550b747ab2ff2d4f WatchSource:0}: Error finding container 11a9ba557cd56428a09c535912c678ba91074e2aaa9f3cd8550b747ab2ff2d4f: Status 404 returned error can't find the container with id 11a9ba557cd56428a09c535912c678ba91074e2aaa9f3cd8550b747ab2ff2d4f
Jul 17 14:49:52.773146 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:52.773126    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0054df8e_6154_4982_8267_2c92e3895772.slice/crio-b9adeddacd0c11f4790d07a1d039a45e7797dfd1b7c77c4e774c0ad4b48aa72d WatchSource:0}: Error finding container b9adeddacd0c11f4790d07a1d039a45e7797dfd1b7c77c4e774c0ad4b48aa72d: Status 404 returned error can't find the container with id b9adeddacd0c11f4790d07a1d039a45e7797dfd1b7c77c4e774c0ad4b48aa72d
Jul 17 14:49:52.774582 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:49:52.774558    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58562432_e34f_4a93_b92a_0a73d3e0e3d1.slice/crio-97dd13792e678cf3e627c479252efed95e6ce3a86f72fdcfd5d7bc5f08ba7fe0 WatchSource:0}: Error finding container 97dd13792e678cf3e627c479252efed95e6ce3a86f72fdcfd5d7bc5f08ba7fe0: Status 404 returned error can't find the container with id 97dd13792e678cf3e627c479252efed95e6ce3a86f72fdcfd5d7bc5f08ba7fe0
Jul 17 14:49:52.878473 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.878441    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 14:44:51 +0000 UTC" deadline="2028-01-20 23:17:14.29026588 +0000 UTC"
Jul 17 14:49:52.878473 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.878470    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13256h27m21.411799331s"
Jul 17 14:49:52.944048 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.944014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:52.944239 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:52.944170    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:49:52.951997 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.951964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qzw8g" event={"ID":"0054df8e-6154-4982-8267-2c92e3895772","Type":"ContainerStarted","Data":"b9adeddacd0c11f4790d07a1d039a45e7797dfd1b7c77c4e774c0ad4b48aa72d"}
Jul 17 14:49:52.953096 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.953054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qmhfb" event={"ID":"93e485cf-83b7-43d4-99bd-d9882c233d5b","Type":"ContainerStarted","Data":"11a9ba557cd56428a09c535912c678ba91074e2aaa9f3cd8550b747ab2ff2d4f"}
Jul 17 14:49:52.954192 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.954170    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mtglb" event={"ID":"e79ace2d-20ee-49ab-8d5d-c465f24d0c76","Type":"ContainerStarted","Data":"41911e8389a854b4c2305220685d4f58991c6e48b66c0628f2ec1b7af884a4e8"}
Jul 17 14:49:52.955779 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.955759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal" event={"ID":"5708bb1a977bc61e2607d3ed9ee9c042","Type":"ContainerStarted","Data":"b00cfb7a93332009a4fb33fad0b04394d10cf2054e5cd15491564c2a98d3b541"}
Jul 17 14:49:52.956748 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.956731    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n" event={"ID":"58562432-e34f-4a93-b92a-0a73d3e0e3d1","Type":"ContainerStarted","Data":"97dd13792e678cf3e627c479252efed95e6ce3a86f72fdcfd5d7bc5f08ba7fe0"}
Jul 17 14:49:52.975157 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:52.975107    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-81.ec2.internal" podStartSLOduration=1.975088871 podStartE2EDuration="1.975088871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:52.973697589 +0000 UTC m=+3.540728720" watchObservedRunningTime="2026-07-17 14:49:52.975088871 +0000 UTC m=+3.542120004"
Jul 17 14:49:53.437887 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:53.437800    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:53.438111 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:53.437954    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:53.438111 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:53.438022    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:49:55.438003009 +0000 UTC m=+6.005034132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:53.878706 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:53.878589    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 14:44:51 +0000 UTC" deadline="2028-03-05 07:30:20.842357995 +0000 UTC"
Jul 17 14:49:53.878706 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:53.878628    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14320h40m26.963734514s"
Jul 17 14:49:53.962616 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:53.962569    2564 generic.go:356] "Generic (PLEG): container finished" podID="a178bbce5cd9e6ffa79c6b7438d62c16" containerID="4e4b11013d2a1d2145adb3a216656c8cddc4803af8158614e6410984431ae139" exitCode=0
Jul 17 14:49:53.963169 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:53.962949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal" event={"ID":"a178bbce5cd9e6ffa79c6b7438d62c16","Type":"ContainerDied","Data":"4e4b11013d2a1d2145adb3a216656c8cddc4803af8158614e6410984431ae139"}
Jul 17 14:49:54.944112 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:54.944077    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:54.944590 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:54.944214    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:49:55.451544 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:55.451032    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:55.451544 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:55.451210    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:55.451544 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:55.451297    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:49:59.451276915 +0000 UTC m=+10.018308038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:55.968150 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:55.968108    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal" event={"ID":"a178bbce5cd9e6ffa79c6b7438d62c16","Type":"ContainerStarted","Data":"08cb4dbb9f59dc628fba1c7345183fc3357bd95277841f2369c1da772a0c3b1e"}
Jul 17 14:49:55.969672 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:55.969640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n" event={"ID":"58562432-e34f-4a93-b92a-0a73d3e0e3d1","Type":"ContainerStarted","Data":"5266823d8a6ad381019814e9bfaf76fffde71f510fdbc126c61ebde88592be0e"}
Jul 17 14:49:55.971144 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:55.971119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qmhfb" event={"ID":"93e485cf-83b7-43d4-99bd-d9882c233d5b","Type":"ContainerStarted","Data":"56b147a8d34b8980b621aa6fcf176092a8f264336aff036dac39d934f454c4e7"}
Jul 17 14:49:55.972751 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:55.972724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mtglb" event={"ID":"e79ace2d-20ee-49ab-8d5d-c465f24d0c76","Type":"ContainerStarted","Data":"912bf7017c7703d417e0a1b8013be8042be232d7c1be6487b3a1ead53de89e62"}
Jul 17 14:49:56.028638 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.028580    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-81.ec2.internal" podStartSLOduration=5.028559874 podStartE2EDuration="5.028559874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:55.990450061 +0000 UTC m=+6.557481193" watchObservedRunningTime="2026-07-17 14:49:56.028559874 +0000 UTC m=+6.595590986"
Jul 17 14:49:56.057360 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.057310    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qmhfb" podStartSLOduration=3.260688869 podStartE2EDuration="6.057288125s" totalImagesPullingTime="2.796599256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:56.029031299 +0000 UTC m=+6.596062432" watchObservedRunningTime="2026-07-17 14:49:56.057288125 +0000 UTC m=+6.624319259"
Jul 17 14:49:56.216552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.216521    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:56.217238 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.217215    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:49:56.244011 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.243908    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mtglb" podStartSLOduration=3.451933799 podStartE2EDuration="6.243892323s" totalImagesPullingTime="2.791958524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:56.057700972 +0000 UTC m=+6.624732105" watchObservedRunningTime="2026-07-17 14:49:56.243892323 +0000 UTC m=+6.810923456"
Jul 17 14:49:56.943920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:56.943885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:56.944131 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:56.944033    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:49:57.978075 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:57.977967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qzw8g" event={"ID":"0054df8e-6154-4982-8267-2c92e3895772","Type":"ContainerStarted","Data":"60f1cbfe54a41788cd9659be9bd37b356defb73b09f066a9bd371c254c9aa995"}
Jul 17 14:49:57.978075 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:57.977994    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 14:49:58.150448 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.150428    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 14:49:58.873644 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.873544    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T14:49:58.150443476Z","UUID":"0ef10a08-2e62-48a3-9612-0649760989b2","Handler":null,"Name":"","Endpoint":""}
Jul 17 14:49:58.875366 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.875348    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 14:49:58.875455 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.875391    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 14:49:58.944053 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.944027    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:58.944199 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:58.944146    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:49:58.982290 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.982264    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n" event={"ID":"58562432-e34f-4a93-b92a-0a73d3e0e3d1","Type":"ContainerStarted","Data":"bc543d0e28d4953e844d65670c6c89e363d8dbb8ec76c8fc2c53b02595bc5fbd"}
Jul 17 14:49:58.982680 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:58.982321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n" event={"ID":"58562432-e34f-4a93-b92a-0a73d3e0e3d1","Type":"ContainerStarted","Data":"84d5397c841671bb6d71bcd1af8772b588d3747a1b15cee753614b19cba13a34"}
Jul 17 14:49:59.007109 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:59.007051    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sz29n" podStartSLOduration=3.191187487 podStartE2EDuration="9.007037665s" totalImagesPullingTime="5.815850178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:59.006974955 +0000 UTC m=+9.574006087" watchObservedRunningTime="2026-07-17 14:49:59.007037665 +0000 UTC m=+9.574068796"
Jul 17 14:49:59.007228 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:59.007192    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qzw8g" podStartSLOduration=4.164981911 podStartE2EDuration="9.007183634s" totalImagesPullingTime="4.842201723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:49:58.008945157 +0000 UTC m=+8.575976290" watchObservedRunningTime="2026-07-17 14:49:59.007183634 +0000 UTC m=+9.574214768"
Jul 17 14:49:59.472862 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:49:59.472832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:49:59.473025 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:59.472927    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:49:59.473025 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:49:59.472975    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:50:07.472961732 +0000 UTC m=+18.039992845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:00.944769 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:00.944726    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:00.945274 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:00.944859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:02.850523 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:02.850488    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:50:02.850939 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:02.850601    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 17 14:50:02.851024 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:02.851011    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mtglb"
Jul 17 14:50:02.944424 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:02.944394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:02.944576 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:02.944480    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:04.944797 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:04.944766    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:04.945190 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:04.944859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:06.944115 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:06.944085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:06.944477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:06.944196    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:07.518674 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:07.518638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:07.518850 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:07.518749    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:07.518850 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:07.518809    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:50:23.518793274 +0000 UTC m=+34.085824387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:08.944434 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:08.944403    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:08.944784 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:08.944498    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:10.943922 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:10.943876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:10.944322 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:10.943997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:12.944683 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:12.944651    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:12.945129 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:12.944747    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:14.944568 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:14.944527    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:14.944936 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:14.944628    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:16.944301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:16.944275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:16.944649 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:16.944369    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:18.944763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:18.944728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:18.945156 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:18.944835    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:19.875519 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.875490    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-dhxhn"]
Jul 17 14:50:19.877408 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.877389    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.879811 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.879770    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 14:50:19.879909 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.879776    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 14:50:19.879909 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.879822    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-95ht5\""
Jul 17 14:50:19.880871 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.880856    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 14:50:19.880918 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.880892    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 14:50:19.996222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-k8s-cni-cncf-io\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996232    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-netns\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-conf-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-daemon-config\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-cnibin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-os-release\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-cni-binary-copy\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-multus\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-kubelet\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-hostroot\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-bin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-etc-kubernetes\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-system-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-socket-dir-parent\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-multus-certs\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:19.996624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:19.996603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkjj5\" (UniqueName: \"kubernetes.io/projected/4deb8705-1e5b-49d4-8daf-329577a363eb-kube-api-access-hkjj5\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.071306 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.071271    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-n6jl8"]
Jul 17 14:50:20.073435 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.073419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.075809 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.075789    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nwxk5\""
Jul 17 14:50:20.076689 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.076672    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 14:50:20.076756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.076725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 14:50:20.097720 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097698    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-cnibin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.097800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097722    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-os-release\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.097800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-cni-binary-copy\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.097800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-multus\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.097800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-kubelet\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.097800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-cnibin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-kubelet\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-os-release\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097852    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-hostroot\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097859    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-multus\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097900    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-bin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097920    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-etc-kubernetes\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-hostroot\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-system-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-var-lib-cni-bin\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-socket-dir-parent\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.097998    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-etc-kubernetes\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098022    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-multus-certs\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-system-cni-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-socket-dir-parent\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkjj5\" (UniqueName: \"kubernetes.io/projected/4deb8705-1e5b-49d4-8daf-329577a363eb-kube-api-access-hkjj5\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098099    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-multus-certs\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098101    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-k8s-cni-cncf-io\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-k8s-cni-cncf-io\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-netns\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-conf-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-daemon-config\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098204    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-host-run-netns\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-conf-dir\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098470    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-cni-binary-copy\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.098857 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.098556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4deb8705-1e5b-49d4-8daf-329577a363eb-multus-daemon-config\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.112447 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.112292    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkjj5\" (UniqueName: \"kubernetes.io/projected/4deb8705-1e5b-49d4-8daf-329577a363eb-kube-api-access-hkjj5\") pod \"multus-dhxhn\" (UID: \"4deb8705-1e5b-49d4-8daf-329577a363eb\") " pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.186211 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.186144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dhxhn"
Jul 17 14:50:20.199223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199201    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199231    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-os-release\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlvl6\" (UniqueName: \"kubernetes.io/projected/0c357674-7536-4047-81c1-84d8e428e8f9-kube-api-access-tlvl6\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199357    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-cnibin\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.199443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.199376    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300411 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300411 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300411    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-os-release\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300431    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvl6\" (UniqueName: \"kubernetes.io/projected/0c357674-7536-4047-81c1-84d8e428e8f9-kube-api-access-tlvl6\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300447    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300506    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-os-release\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-system-cni-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-cnibin\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300688    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-cnibin\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300696    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.300798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.300662    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c357674-7536-4047-81c1-84d8e428e8f9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.301039 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.301021    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-binary-copy\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.301092 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.301040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.301173 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.301159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c357674-7536-4047-81c1-84d8e428e8f9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.311707 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.311683    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlvl6\" (UniqueName: \"kubernetes.io/projected/0c357674-7536-4047-81c1-84d8e428e8f9-kube-api-access-tlvl6\") pod \"multus-additional-cni-plugins-n6jl8\" (UID: \"0c357674-7536-4047-81c1-84d8e428e8f9\") " pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.380974 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.380943    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n6jl8"
Jul 17 14:50:20.389598 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:50:20.389571    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c357674_7536_4047_81c1_84d8e428e8f9.slice/crio-75e9c304d91b50d41dabe38f6f847ac1377a66fa02161a6524fb46e1a823b693 WatchSource:0}: Error finding container 75e9c304d91b50d41dabe38f6f847ac1377a66fa02161a6524fb46e1a823b693: Status 404 returned error can't find the container with id 75e9c304d91b50d41dabe38f6f847ac1377a66fa02161a6524fb46e1a823b693
Jul 17 14:50:20.864276 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.864243    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-c8d6d"]
Jul 17 14:50:20.866374 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.866358    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:20.866446 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:20.866429    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:20.943856 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:20.943825    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:20.944089 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:20.943969    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:21.005949 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.005871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr6tv\" (UniqueName: \"kubernetes.io/projected/ea32a2a8-657e-498b-a573-0c85ffa9eec3-kube-api-access-hr6tv\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.005949 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.005920    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.017478 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.017444    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerStarted","Data":"75e9c304d91b50d41dabe38f6f847ac1377a66fa02161a6524fb46e1a823b693"}
Jul 17 14:50:21.018729 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.018698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dhxhn" event={"ID":"4deb8705-1e5b-49d4-8daf-329577a363eb","Type":"ContainerStarted","Data":"0b19ef26812e603a7cd8fe846a9dcca96b36f7609500e427bb35d3b0f5407863"}
Jul 17 14:50:21.107001 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.106965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.107194 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.107053    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr6tv\" (UniqueName: \"kubernetes.io/projected/ea32a2a8-657e-498b-a573-0c85ffa9eec3-kube-api-access-hr6tv\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.107194 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:21.107176    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:21.107301 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:21.107253    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:21.607232741 +0000 UTC m=+32.174263857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:21.118243 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.118182    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr6tv\" (UniqueName: \"kubernetes.io/projected/ea32a2a8-657e-498b-a573-0c85ffa9eec3-kube-api-access-hr6tv\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.611177 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:21.611051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:21.611373 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:21.611241    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:21.611373 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:21.611311    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:22.611293177 +0000 UTC m=+33.178324291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:22.619598 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:22.619566    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:22.620090 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:22.619695    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:22.620090 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:22.619762    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:24.619743588 +0000 UTC m=+35.186774714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:22.944528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:22.944452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:22.944528 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:22.944482    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:22.944714 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:22.944578    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:22.944771 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:22.944717    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:23.024263 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:23.024230    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="fabf800df90a8f8b5270720abd238a62b379238f3419518090035150f36ef0c1" exitCode=0
Jul 17 14:50:23.024442 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:23.024279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"fabf800df90a8f8b5270720abd238a62b379238f3419518090035150f36ef0c1"}
Jul 17 14:50:23.528228 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:23.528193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:23.528411 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:23.528365    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:23.528474 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:23.528430    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:50:55.528410356 +0000 UTC m=+66.095441481 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:24.635124 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:24.635088    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:24.635756 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:24.635304    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:24.635756 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:24.635382    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:28.635359953 +0000 UTC m=+39.202391079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:24.944663 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:24.944580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:24.944663 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:24.944628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:24.944857 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:24.944744    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:24.944924 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:24.944896    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:26.944532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:26.944497    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:26.944532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:26.944523    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:26.945054 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:26.944651    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:26.945054 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:26.944658    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:28.661837 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:28.661788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:28.662345 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:28.661952    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:28.662345 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:28.662035    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:36.662014294 +0000 UTC m=+47.229045408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:28.944019 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:28.943941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:28.944202 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:28.944084    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:28.944202 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:28.943941    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:28.944315 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:28.944220    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:30.773650 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.773612    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-q9nq6"]
Jul 17 14:50:30.776984 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.776960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.780049 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.780016    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 14:50:30.780049 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.780036    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 14:50:30.780227 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.780097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 14:50:30.780227 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.780100    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 14:50:30.780227 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.780036    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 14:50:30.781226 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.781206    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 14:50:30.781320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.781274    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6r5mr\""
Jul 17 14:50:30.876885 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.876439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.876902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.876930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.876956    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.876985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbs2l\" (UniqueName: \"kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877082    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877122    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877153    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877726 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877366    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877726 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877726 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877403    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.877726 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.877441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.944440 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.944409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:30.944592 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.944542    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:30.944649 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:30.944586    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:30.944649 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:30.944625    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:30.978301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978452 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978452 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978452 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978452 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978420    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978452 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978497    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978525    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978725    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbs2l\" (UniqueName: \"kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.978763 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978965    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.978989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979331    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979402    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979444    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979451    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979495    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.979854 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.980193 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.979883    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.981265 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.981243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:30.989907 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:30.989885    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbs2l\" (UniqueName: \"kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l\") pod \"ovnkube-node-q9nq6\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") " pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:31.086472 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:31.086416    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:31.444482 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:50:31.444447    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cdcb6d_2d42_4b20_80d6_4fb3914a92c4.slice/crio-24a486f32d73cbd4686d3a34568a3a14f61f5f9ed9057c5bedba3f20dc2f14b1 WatchSource:0}: Error finding container 24a486f32d73cbd4686d3a34568a3a14f61f5f9ed9057c5bedba3f20dc2f14b1: Status 404 returned error can't find the container with id 24a486f32d73cbd4686d3a34568a3a14f61f5f9ed9057c5bedba3f20dc2f14b1
Jul 17 14:50:32.044844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.044545    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"24a486f32d73cbd4686d3a34568a3a14f61f5f9ed9057c5bedba3f20dc2f14b1"}
Jul 17 14:50:32.046592 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.046566    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="4506884eac76c9a29b75e4df25d761873e5b83de536e21611e385396cb8fd901" exitCode=0
Jul 17 14:50:32.046724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.046644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"4506884eac76c9a29b75e4df25d761873e5b83de536e21611e385396cb8fd901"}
Jul 17 14:50:32.048420 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.048390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dhxhn" event={"ID":"4deb8705-1e5b-49d4-8daf-329577a363eb","Type":"ContainerStarted","Data":"691d21ce36d1f7eae5ab35b155d5b11dc70cc09f00da42057ca4ab8e1c90e7a4"}
Jul 17 14:50:32.101509 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.101447    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dhxhn" podStartSLOduration=1.804524845 podStartE2EDuration="13.101426733s" totalImagesPullingTime="11.296901888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:50:32.099969397 +0000 UTC m=+42.667000532" watchObservedRunningTime="2026-07-17 14:50:32.101426733 +0000 UTC m=+42.668457863"
Jul 17 14:50:32.943992 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.943957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:32.944169 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:32.943957    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:32.944169 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:32.944111    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:32.944300 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:32.944274    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:33.691053 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:33.691017    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-xqvs7"]
Jul 17 14:50:33.695989 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:33.695959    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:33.696133 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:33.696049    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:33.799239 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:33.799212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:33.900200 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:33.900166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:33.909250 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:33.909218    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:33.909250 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:33.909243    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:33.909250 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:33.909254    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:33.909489 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:33.909323    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:34.409304298 +0000 UTC m=+44.976335412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:34.055740 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:34.055654    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="19b3bf7dce87872e5d83121fa3e0b208f5bf1791a87fd4dd2dee1fa1075eb07b" exitCode=0
Jul 17 14:50:34.055740 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:34.055707    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"19b3bf7dce87872e5d83121fa3e0b208f5bf1791a87fd4dd2dee1fa1075eb07b"}
Jul 17 14:50:34.504959 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:34.504923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:34.505152 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.505136    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:34.505235 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.505160    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:34.505235 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.505175    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:34.505333 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.505237    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:35.505218439 +0000 UTC m=+46.072249561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:34.944365 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:34.944339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:34.944719 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:34.944339    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:34.944719 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.944430    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:34.944719 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:34.944498    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:35.059944 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:35.059852    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="0939a8d26d82b557a82cbd7a4cc8a7a0fe7ba255fdb83822c3661e58a9cc6b22" exitCode=0
Jul 17 14:50:35.059944 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:35.059913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"0939a8d26d82b557a82cbd7a4cc8a7a0fe7ba255fdb83822c3661e58a9cc6b22"}
Jul 17 14:50:35.512446 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:35.512406    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:35.512643 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:35.512625    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:35.512717 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:35.512654    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:35.512717 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:35.512668    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:35.512811 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:35.512740    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:37.512719845 +0000 UTC m=+48.079750967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:35.944884 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:35.944395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:35.944884 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:35.944537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:36.721499 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:36.721455    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:36.721668 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:36.721589    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:36.721668 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:36.721654    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:52.721634897 +0000 UTC m=+63.288666014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:36.944600 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:36.944271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:36.944600 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:36.944304    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:36.944600 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:36.944427    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:36.944600 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:36.944509    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:37.073032 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.073005    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-dkv72"]
Jul 17 14:50:37.077288 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.077269    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.082471 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.082429    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:50:37.083348 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.083329    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fsd6h\""
Jul 17 14:50:37.083465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.083369    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 14:50:37.083465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.083457    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 14:50:37.225740 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.225703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58dm5\" (UniqueName: \"kubernetes.io/projected/f9d66577-def7-4626-82ad-df0133c063d8-kube-api-access-58dm5\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.225916 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.225750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9d66577-def7-4626-82ad-df0133c063d8-iptables-alerter-script\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.225916 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.225862    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9d66577-def7-4626-82ad-df0133c063d8-host-slash\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.327141 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.327030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58dm5\" (UniqueName: \"kubernetes.io/projected/f9d66577-def7-4626-82ad-df0133c063d8-kube-api-access-58dm5\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.327141 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.327099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9d66577-def7-4626-82ad-df0133c063d8-iptables-alerter-script\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.327351 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.327161    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9d66577-def7-4626-82ad-df0133c063d8-host-slash\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.327351 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.327243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f9d66577-def7-4626-82ad-df0133c063d8-host-slash\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.327774 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.327748    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f9d66577-def7-4626-82ad-df0133c063d8-iptables-alerter-script\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.339842 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.339810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58dm5\" (UniqueName: \"kubernetes.io/projected/f9d66577-def7-4626-82ad-df0133c063d8-kube-api-access-58dm5\") pod \"iptables-alerter-dkv72\" (UID: \"f9d66577-def7-4626-82ad-df0133c063d8\") " pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.388463 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.388209    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dkv72"
Jul 17 14:50:37.398252 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:50:37.398187    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d66577_def7_4626_82ad_df0133c063d8.slice/crio-7f737711af51f8c443017762b851e38ed93f68e392cd12505e50de16c984cb00 WatchSource:0}: Error finding container 7f737711af51f8c443017762b851e38ed93f68e392cd12505e50de16c984cb00: Status 404 returned error can't find the container with id 7f737711af51f8c443017762b851e38ed93f68e392cd12505e50de16c984cb00
Jul 17 14:50:37.528480 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.528439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:37.528643 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:37.528620    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:37.528708 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:37.528645    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:37.528708 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:37.528659    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:37.528823 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:37.528720    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:41.528702395 +0000 UTC m=+52.095733509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:37.944302 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:37.944265    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:37.944487 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:37.944460    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:38.067779 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:38.067671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dkv72" event={"ID":"f9d66577-def7-4626-82ad-df0133c063d8","Type":"ContainerStarted","Data":"7f737711af51f8c443017762b851e38ed93f68e392cd12505e50de16c984cb00"}
Jul 17 14:50:38.944057 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:38.944020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:38.944057 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:38.944054    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:38.944598 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:38.944171    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:38.944598 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:38.944305    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:39.945197 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:39.945169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:39.945588 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:39.945251    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:40.944693 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:40.944657    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:40.944859 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:40.944798    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:40.944859 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:40.944845    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:40.944983 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:40.944913    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:41.559618 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:41.559575    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:41.560113 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:41.559773    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:41.560113 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:41.559801    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:41.560113 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:41.559815    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:41.560113 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:41.559881    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:50:49.559864515 +0000 UTC m=+60.126895643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:41.944756 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:41.944723    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:41.944936 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:41.944859    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:42.944456 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:42.944410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:42.944906 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:42.944560    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:42.944906 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:42.944572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:42.944906 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:42.944688    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:43.944568 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:43.944530    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:43.944999 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:43.944657    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:44.944641 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:44.944616    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:44.944973 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:44.944646    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:44.944973 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:44.944764    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:44.944973 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:44.944899    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:45.944652 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:45.944428    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:45.945035 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:45.944669    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:46.089320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.089297    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="fe20fe166955d78baaf79c95612c77a1724e39073179d2bae337ca10730a6e81" exitCode=0
Jul 17 14:50:46.089414 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.089367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"fe20fe166955d78baaf79c95612c77a1724e39073179d2bae337ca10730a6e81"}
Jul 17 14:50:46.092324 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092304    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/0.log"
Jul 17 14:50:46.092636 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092618    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="97eece9c03d5d668753220b6095d872ab794e381f88a4a6aabd9dad035e57763" exitCode=1
Jul 17 14:50:46.092702 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092651    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"}
Jul 17 14:50:46.092738 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092704    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"}
Jul 17 14:50:46.092738 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"}
Jul 17 14:50:46.092796 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"}
Jul 17 14:50:46.092796 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"97eece9c03d5d668753220b6095d872ab794e381f88a4a6aabd9dad035e57763"}
Jul 17 14:50:46.092855 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.092817    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"}
Jul 17 14:50:46.944629 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.944593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:46.944805 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:46.944697    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:46.944805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:46.944753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:46.945279 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:46.944844    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:47.096630 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:47.096594    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dkv72" event={"ID":"f9d66577-def7-4626-82ad-df0133c063d8","Type":"ContainerStarted","Data":"b62df7d0f6a1f3a4cc98f97a9aa918180055aef9771582b8f3992ccbf21047ee"}
Jul 17 14:50:47.099576 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:47.099549    2564 generic.go:356] "Generic (PLEG): container finished" podID="0c357674-7536-4047-81c1-84d8e428e8f9" containerID="ce740fa02623647d76ce21d580120e62919c03d536b570c38acfe75c3e286669" exitCode=0
Jul 17 14:50:47.099714 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:47.099586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerDied","Data":"ce740fa02623647d76ce21d580120e62919c03d536b570c38acfe75c3e286669"}
Jul 17 14:50:47.136411 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:47.136361    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dkv72" podStartSLOduration=2.481956728 podStartE2EDuration="10.136342042s" totalImagesPullingTime="7.654385314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:50:47.112946884 +0000 UTC m=+57.679978016" watchObservedRunningTime="2026-07-17 14:50:47.136342042 +0000 UTC m=+57.703373175"
Jul 17 14:50:47.944790 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:47.944768    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:47.944906 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:47.944861    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:48.105160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.105134    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/0.log"
Jul 17 14:50:48.105532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.105511    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"}
Jul 17 14:50:48.108459 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.108436    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" event={"ID":"0c357674-7536-4047-81c1-84d8e428e8f9","Type":"ContainerStarted","Data":"fcfd3a3ee64649ac5cba41a8893112cccd73e6ac192a32e2eef3df6112e12cfb"}
Jul 17 14:50:48.131656 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.131609    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n6jl8" podStartSLOduration=5.078339384 podStartE2EDuration="28.131594365s" totalImagesPullingTime="23.053254981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:50:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:50:48.131206552 +0000 UTC m=+58.698237684" watchObservedRunningTime="2026-07-17 14:50:48.131594365 +0000 UTC m=+58.698625496"
Jul 17 14:50:48.944542 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.944511    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:48.944542 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:48.944528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:48.944750 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:48.944622    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:48.944750 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:48.944742    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:49.615858 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:49.615821    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:49.616534 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:49.615946    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:50:49.616534 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:49.615959    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:50:49.616534 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:49.615967    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:49.616534 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:49.616014    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:05.616001138 +0000 UTC m=+76.183032248 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:50:49.944852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:49.944751    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:49.945023 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:49.944847    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:50.116600 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.116324    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/0.log"
Jul 17 14:50:50.117103 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.116870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"}
Jul 17 14:50:50.117477 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.117214    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:50.117477 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.117269    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:50.117477 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.117320    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:50.117477 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.117422    2564 scope.go:117] "RemoveContainer" containerID="97eece9c03d5d668753220b6095d872ab794e381f88a4a6aabd9dad035e57763"
Jul 17 14:50:50.135953 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.135671    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:50.140834 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.140488    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:50:50.944224 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.944182    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:50.944224 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:50.944219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:50.944742 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:50.944302    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:50.944742 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:50.944415    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:51.122593 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:51.122566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/0.log"
Jul 17 14:50:51.122891 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:51.122869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerStarted","Data":"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"}
Jul 17 14:50:51.150838 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:51.150793    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podStartSLOduration=7.49313193 podStartE2EDuration="21.150778611s" totalImagesPullingTime="13.657646681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:50:51.150503331 +0000 UTC m=+61.717534483" watchObservedRunningTime="2026-07-17 14:50:51.150778611 +0000 UTC m=+61.717809743"
Jul 17 14:50:51.944078 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:51.944032    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:51.944240 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:51.944146    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:52.739920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:52.739886    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:52.740306 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:52.740001    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:52.740306 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:52.740053    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:24.740040128 +0000 UTC m=+95.307071238 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:50:52.943863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:52.943832    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:52.943863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:52.943854    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:52.944042 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:52.943954    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:52.944112 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:52.944093    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:53.944415 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:53.944383    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:53.944819 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:53.944480    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:54.944171 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:54.944135    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:54.944347 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:54.944144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:54.944347 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:54.944249    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:54.944347 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:54.944306    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:55.518462 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518432    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-q9nq6"]
Jul 17 14:50:55.518834 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518694    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-controller" containerID="cri-o://2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2" gracePeriod=30
Jul 17 14:50:55.518834 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518719    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging" containerID="cri-o://c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c" gracePeriod=30
Jul 17 14:50:55.518834 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518767    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-ovn-metrics" containerID="cri-o://239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754" gracePeriod=30
Jul 17 14:50:55.519000 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518847    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-node" containerID="cri-o://1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5" gracePeriod=30
Jul 17 14:50:55.519000 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518719    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb" containerID="cri-o://9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" gracePeriod=30
Jul 17 14:50:55.519000 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518801    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="northd" containerID="cri-o://ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85" gracePeriod=30
Jul 17 14:50:55.519000 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.518904    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb" containerID="cri-o://baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" gracePeriod=30
Jul 17 14:50:55.544344 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.544296    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 17 14:50:55.555228 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.555196    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller" containerID="cri-o://b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5" gracePeriod=30
Jul 17 14:50:55.558836 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.558812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:55.558946 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:55.558934    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:55.558995 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:55.558986    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret podName:a3f0b694-edc9-4e75-81de-7b528fa2195f nodeName:}" failed. No retries permitted until 2026-07-17 14:51:59.55897125 +0000 UTC m=+130.126002360 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret") pod "global-pull-secret-syncer-6hdkd" (UID: "a3f0b694-edc9-4e75-81de-7b528fa2195f") : object "kube-system"/"original-pull-secret" not registered
Jul 17 14:50:55.944212 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:55.944181    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:55.944369 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:55.944291    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:56.135740 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.135712    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/1.log"
Jul 17 14:50:56.137297 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.137278    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 17 14:50:56.137673 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.137660    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-node/0.log"
Jul 17 14:50:56.138026 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138013    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/0.log"
Jul 17 14:50:56.138422 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138408    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-controller/0.log"
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138440    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c" exitCode=143
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138453    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" exitCode=0
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138459    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" exitCode=0
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138464    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85" exitCode=0
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138469    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754" exitCode=143
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138473    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5" exitCode=143
Jul 17 14:50:56.138479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138478    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2" exitCode=143
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"}
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138567    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"}
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"}
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138625    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"}
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138637    2564 scope.go:117] "RemoveContainer" containerID="97eece9c03d5d668753220b6095d872ab794e381f88a4a6aabd9dad035e57763"
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"}
Jul 17 14:50:56.138696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"}
Jul 17 14:50:56.138931 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.138703    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"}
Jul 17 14:50:56.944120 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.944081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:56.944517 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:56.944081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:56.944517 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:56.944196    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:56.944517 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:56.944260    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:57.142546 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:57.142514    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/1.log"
Jul 17 14:50:57.144137 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:57.144120    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 17 14:50:57.144522 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:57.144507    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-node/0.log"
Jul 17 14:50:57.144913 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:57.144901    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-controller/0.log"
Jul 17 14:50:57.944283 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:57.944257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:57.944655 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:57.944358    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:50:58.944805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:58.944774    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:50:58.945205 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:58.944870    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:50:58.945205 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:58.944922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:50:58.945205 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:58.945005    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:50:59.945414 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:50:59.945385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:50:59.945811 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:50:59.945458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:00.118408 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118226    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:00.118589 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118224    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:00.118650 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118626    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:00.118711 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118677    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:00.118859 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118837    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:00.118911 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118869    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb" probeResult="unknown"
Jul 17 14:51:00.118911 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118890    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:00.118990 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.118919    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb" probeResult="unknown"
Jul 17 14:51:00.944105 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:00.944059    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:00.944268 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:00.944080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:00.944268 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.944159    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:00.944378 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:00.944262    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:01.944727 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:01.944686    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:01.945235 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:01.944788    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:02.944692 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:02.944645    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:02.944874 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:02.944647    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:02.944874 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:02.944764    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:02.944874 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:02.944856    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:03.944372 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:03.944341    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:03.944539 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:03.944443    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:04.944832 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:04.944802    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:04.944832 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:04.944802    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:04.945259 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:04.945160    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:04.945299 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:04.945258    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:05.634800 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:05.634771    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:05.634971 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:05.634884    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 14:51:05.634971 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:05.634897    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 14:51:05.634971 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:05.634905    2564 projected.go:196] Error preparing data for projected volume kube-api-access-b869d for pod openshift-network-diagnostics/network-check-target-xqvs7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:51:05.634971 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:05.634950    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d podName:c2616f3d-01dc-4944-8fc3-13a91239d615 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.634938697 +0000 UTC m=+108.201969806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b869d" (UniqueName: "kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d") pod "network-check-target-xqvs7" (UID: "c2616f3d-01dc-4944-8fc3-13a91239d615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 14:51:05.944913 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:05.944820    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:05.945406 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:05.944917    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:06.944339 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:06.944310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:06.944479 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:06.944404    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:06.944479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:06.944461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:06.944579 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:06.944545    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:07.944429 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:07.944400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:07.944825 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:07.944498    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:08.944538 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:08.944509    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:08.944921 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:08.944613    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:08.944921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:08.944667    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:08.944921 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:08.944754    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:09.945900 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:09.945871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:09.946287 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:09.945959    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:10.118470 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118427    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:10.118627 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118426    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:10.118735 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118716    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:10.118773 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118720    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:10.119006 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118976    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:10.119006 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.118983    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:10.119195 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.119010    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb" probeResult="unknown"
Jul 17 14:51:10.119195 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.119032    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb" probeResult="unknown"
Jul 17 14:51:10.943886 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:10.943846    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:10.944086 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:10.943860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:10.944086 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.943957    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:10.944086 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:10.944028    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:11.944145 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:11.944116    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:11.944568 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:11.944204    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:12.944350 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:12.944322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:12.944705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:12.944322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:12.944705 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:12.944423    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:12.944705 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:12.944490    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:13.944524 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:13.944494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:13.944975 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:13.944620    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:14.944195 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:14.944168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:14.944374 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:14.944249    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:14.944374 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:14.944275    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:14.944453 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:14.944371    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:15.944121 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:15.944092    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:15.944483 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:15.944191    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:16.944257 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:16.944221    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:16.944257 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:16.944253    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:16.944725 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:16.944335    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:16.944725 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:16.944495    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:17.186502 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:17.186476    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:51:17.186651 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:17.186517    2564 generic.go:356] "Generic (PLEG): container finished" podID="4deb8705-1e5b-49d4-8daf-329577a363eb" containerID="691d21ce36d1f7eae5ab35b155d5b11dc70cc09f00da42057ca4ab8e1c90e7a4" exitCode=1
Jul 17 14:51:17.186651 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:17.186574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dhxhn" event={"ID":"4deb8705-1e5b-49d4-8daf-329577a363eb","Type":"ContainerDied","Data":"691d21ce36d1f7eae5ab35b155d5b11dc70cc09f00da42057ca4ab8e1c90e7a4"}
Jul 17 14:51:17.186920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:17.186907    2564 scope.go:117] "RemoveContainer" containerID="691d21ce36d1f7eae5ab35b155d5b11dc70cc09f00da42057ca4ab8e1c90e7a4"
Jul 17 14:51:17.943989 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:17.943946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:17.944182 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:17.944090    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:18.190305 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:18.190280    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:51:18.190718 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:18.190326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dhxhn" event={"ID":"4deb8705-1e5b-49d4-8daf-329577a363eb","Type":"ContainerStarted","Data":"a6fcde222e159e14f6849fcc165d7322670f3025d4e84206004f2630ea97078c"}
Jul 17 14:51:18.944657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:18.944623    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:18.944853 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:18.944727    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:18.944853 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:18.944785    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:18.944997 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:18.944889    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:19.945614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:19.945585    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:19.945997 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:19.945663    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:20.117493 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117445    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:20.117493 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117474    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:20.117732 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117712    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:20.117786 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117727    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:20.117996 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117971    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"sb\"\n"]
Jul 17 14:51:20.118056 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.118009    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb" probeResult="unknown"
Jul 17 14:51:20.118107 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.117975    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073" cmd=["/bin/bash","-c","set -xeo pipefail\n. /ovnkube-lib/ovnkube-lib.sh || exit 1\novndb-readiness-probe \"nb\"\n"]
Jul 17 14:51:20.118144 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.118101    2564 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 is running failed: container process not found" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb" probeResult="unknown"
Jul 17 14:51:20.944076 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:20.944044    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:20.944230 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.944163    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:20.944230 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:20.944216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:20.944316 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:20.944298    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:21.944327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:21.944295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:21.944711 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:21.944496    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:22.944428 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:22.944391    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:22.944428 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:22.944413    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:22.944925 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:22.944509    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:22.944925 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:22.944617    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:23.944886 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:23.944851    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:23.945336 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:23.944967    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:24.774304 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:24.774272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:24.774465 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:24.774407    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:51:24.774504 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:24.774471    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs podName:ea32a2a8-657e-498b-a573-0c85ffa9eec3 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:28.774455639 +0000 UTC m=+159.341486749 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs") pod "network-metrics-daemon-c8d6d" (UID: "ea32a2a8-657e-498b-a573-0c85ffa9eec3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 14:51:24.944540 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:24.944489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:24.944540 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:24.944536    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:24.944796 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:24.944633    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:24.944796 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:24.944736    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:25.520318 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:25.520276    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Jul 17 14:51:25.521169 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:25.521135    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Jul 17 14:51:25.521923 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:25.521897    2564 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5" cmd=["test","-f","/etc/cni/net.d/10-ovn-kubernetes.conf"]
Jul 17 14:51:25.522033 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:25.521936    2564 prober.go:104] "Probe errored" err="rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller" probeResult="unknown"
Jul 17 14:51:25.944155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:25.944121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:25.944345 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:25.944243    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:26.082922 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.082904    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/1.log"
Jul 17 14:51:26.083397 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.083383    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovnkube-controller/0.log"
Jul 17 14:51:26.084669 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.084655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 17 14:51:26.085094 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.085079    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-node/0.log"
Jul 17 14:51:26.085495 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.085482    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-controller/0.log"
Jul 17 14:51:26.086790 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.086777    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:51:26.185123 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185101    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185127    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185145    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185164    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185199    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch" (OuterVolumeSpecName: "var-lib-openvswitch") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "var-lib-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185214    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash" (OuterVolumeSpecName: "host-slash") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-slash". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185227    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log" (OuterVolumeSpecName: "node-log") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "node-log". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185199    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet" (OuterVolumeSpecName: "host-kubelet") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-kubelet". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185272    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185286    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185302    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185317    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd" (OuterVolumeSpecName: "host-cni-netd") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-cni-netd". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185330    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units" (OuterVolumeSpecName: "systemd-units") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "systemd-units". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185359    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185380    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185375    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes" (OuterVolumeSpecName: "host-var-lib-cni-networks-ovn-kubernetes") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-var-lib-cni-networks-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185403    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185421    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket" (OuterVolumeSpecName: "log-socket") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "log-socket". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.185444 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185467    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin" (OuterVolumeSpecName: "host-cni-bin") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-cni-bin". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185490    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185503    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch" (OuterVolumeSpecName: "run-openvswitch") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "run-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185516    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185534    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides" (OuterVolumeSpecName: "env-overrides") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "env-overrides". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185560    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbs2l\" (UniqueName: \"kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185545    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns" (OuterVolumeSpecName: "host-run-netns") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-run-netns". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185608    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes\") pod \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\" (UID: \"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4\") "
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185573    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config" (OuterVolumeSpecName: "ovnkube-config") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "ovnkube-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185700    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes" (OuterVolumeSpecName: "host-run-ovn-kubernetes") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "host-run-ovn-kubernetes". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185741    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib" (OuterVolumeSpecName: "ovnkube-script-lib") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "ovnkube-script-lib". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185782    2564 reconciler_common.go:299] "Volume detached for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-kubelet\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185789    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn" (OuterVolumeSpecName: "run-ovn") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "run-ovn". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185802    2564 reconciler_common.go:299] "Volume detached for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-var-lib-openvswitch\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185818    2564 reconciler_common.go:299] "Volume detached for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-slash\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185832    2564 reconciler_common.go:299] "Volume detached for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-config\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch" (OuterVolumeSpecName: "etc-openvswitch") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "etc-openvswitch". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185846    2564 reconciler_common.go:299] "Volume detached for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-node-log\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185880    2564 reconciler_common.go:299] "Volume detached for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-env-overrides\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185895    2564 reconciler_common.go:299] "Volume detached for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-netd\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185907    2564 reconciler_common.go:299] "Volume detached for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-systemd-units\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185922    2564 reconciler_common.go:299] "Volume detached for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-var-lib-cni-networks-ovn-kubernetes\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185936    2564 reconciler_common.go:299] "Volume detached for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-log-socket\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185949    2564 reconciler_common.go:299] "Volume detached for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-cni-bin\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185962    2564 reconciler_common.go:299] "Volume detached for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovnkube-script-lib\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185975    2564 reconciler_common.go:299] "Volume detached for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-openvswitch\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.186777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.185987    2564 reconciler_common.go:299] "Volume detached for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-netns\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.190333 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.190296    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l" (OuterVolumeSpecName: "kube-api-access-pbs2l") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "kube-api-access-pbs2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:51:26.190789 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.190769    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd" (OuterVolumeSpecName: "run-systemd") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "run-systemd". PluginName "kubernetes.io/host-path", VolumeGIDValue ""
Jul 17 14:51:26.191309 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.191290    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert" (OuterVolumeSpecName: "ovn-node-metrics-cert") pod "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" (UID: "39cdcb6d-2d42-4b20-80d6-4fb3914a92c4"). InnerVolumeSpecName "ovn-node-metrics-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:51:26.208391 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.208347    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-acl-logging/1.log"
Jul 17 14:51:26.208795 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.208782    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovnkube-controller/0.log"
Jul 17 14:51:26.209956 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.209942    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-ovn-metrics/0.log"
Jul 17 14:51:26.210317 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210305    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/kube-rbac-proxy-node/0.log"
Jul 17 14:51:26.210664 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210652    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-q9nq6_39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/ovn-controller/0.log"
Jul 17 14:51:26.210721 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210685    2564 generic.go:356] "Generic (PLEG): container finished" podID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5" exitCode=137
Jul 17 14:51:26.210755 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"}
Jul 17 14:51:26.210798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210784    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6"
Jul 17 14:51:26.210864 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q9nq6" event={"ID":"39cdcb6d-2d42-4b20-80d6-4fb3914a92c4","Type":"ContainerDied","Data":"24a486f32d73cbd4686d3a34568a3a14f61f5f9ed9057c5bedba3f20dc2f14b1"}
Jul 17 14:51:26.210928 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.210916    2564 scope.go:117] "RemoveContainer" containerID="c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"
Jul 17 14:51:26.217744 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.217728    2564 scope.go:117] "RemoveContainer" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"
Jul 17 14:51:26.224846 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.224830    2564 scope.go:117] "RemoveContainer" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"
Jul 17 14:51:26.230634 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.230621    2564 scope.go:117] "RemoveContainer" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"
Jul 17 14:51:26.235995 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.235979    2564 scope.go:117] "RemoveContainer" containerID="ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"
Jul 17 14:51:26.241372 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.241356    2564 scope.go:117] "RemoveContainer" containerID="239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"
Jul 17 14:51:26.246883 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.246859    2564 scope.go:117] "RemoveContainer" containerID="1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"
Jul 17 14:51:26.252313 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.252295    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-q9nq6"]
Jul 17 14:51:26.252490 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.252475    2564 scope.go:117] "RemoveContainer" containerID="2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"
Jul 17 14:51:26.253977 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.253958    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-lk2h9"]
Jul 17 14:51:26.254155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254137    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-node"
Jul 17 14:51:26.254155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254154    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-node"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254162    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-ovn-metrics"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254168    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-ovn-metrics"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254175    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="northd"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254181    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="northd"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254187    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-controller"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254192    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-controller"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254197    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254202    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254207    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254212    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254218    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254222    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254228    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254232    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254237    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254243    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254270    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="northd"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254277    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254283    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-acl-logging"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254287    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="nbdb"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254294    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovn-controller"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254298    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="sbdb"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254302    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-node"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254306    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="kube-rbac-proxy-ovn-metrics"
Jul 17 14:51:26.254773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.254311    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" containerName="ovnkube-controller"
Jul 17 14:51:26.257733 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.257711    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.258754 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.258737    2564 scope.go:117] "RemoveContainer" containerID="c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"
Jul 17 14:51:26.259007 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.258991    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c\": container with ID starting with c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c not found: ID does not exist" containerID="c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"
Jul 17 14:51:26.259087 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259019    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c"} err="failed to get container status \"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c\": rpc error: code = NotFound desc = could not find container \"c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c\": container with ID starting with c043803f2165d7afee621254d4743fd69298c068631395b6c8ca74f501a8676c not found: ID does not exist"
Jul 17 14:51:26.259087 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259078    2564 scope.go:117] "RemoveContainer" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"
Jul 17 14:51:26.259364 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.259342    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5\": container with ID starting with b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5 not found: ID does not exist" containerID="b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"
Jul 17 14:51:26.259405 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259374    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5"} err="failed to get container status \"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5\": rpc error: code = NotFound desc = could not find container \"b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5\": container with ID starting with b78a2c64012a531434a0d71118c2c26bb63f9f5371c2a12ce49df7420c010cc5 not found: ID does not exist"
Jul 17 14:51:26.259442 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259407    2564 scope.go:117] "RemoveContainer" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"
Jul 17 14:51:26.259683 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.259665    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377\": container with ID starting with 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 not found: ID does not exist" containerID="9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"
Jul 17 14:51:26.259724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259691    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377"} err="failed to get container status \"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377\": rpc error: code = NotFound desc = could not find container \"9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377\": container with ID starting with 9adf9eda431ace1b54575cab0465c52bdf90545dc38cd697ca7f3a76fac7a377 not found: ID does not exist"
Jul 17 14:51:26.259724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259707    2564 scope.go:117] "RemoveContainer" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"
Jul 17 14:51:26.259966 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.259942    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073\": container with ID starting with baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 not found: ID does not exist" containerID="baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"
Jul 17 14:51:26.260004 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259975    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073"} err="failed to get container status \"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073\": rpc error: code = NotFound desc = could not find container \"baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073\": container with ID starting with baee01b8098bd4916c81039d8a5f3521a898f010cda8f0e7f5ba4318ee5ed073 not found: ID does not exist"
Jul 17 14:51:26.260004 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.259993    2564 scope.go:117] "RemoveContainer" containerID="ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"
Jul 17 14:51:26.260277 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.260262    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85\": container with ID starting with ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85 not found: ID does not exist" containerID="ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"
Jul 17 14:51:26.260357 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260281    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85"} err="failed to get container status \"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85\": rpc error: code = NotFound desc = could not find container \"ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85\": container with ID starting with ad98284b2f685678c96ddae56850ab56d59a0c2dd1653d25a26234afb3c48f85 not found: ID does not exist"
Jul 17 14:51:26.260357 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260299    2564 scope.go:117] "RemoveContainer" containerID="239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"
Jul 17 14:51:26.260543 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.260528    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754\": container with ID starting with 239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754 not found: ID does not exist" containerID="239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"
Jul 17 14:51:26.260585 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260547    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754"} err="failed to get container status \"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754\": rpc error: code = NotFound desc = could not find container \"239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754\": container with ID starting with 239243b83e8dbf7e631f085a2e1491d8fbd120b7c71a67c81b349c4a4ef2f754 not found: ID does not exist"
Jul 17 14:51:26.260585 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260561    2564 scope.go:117] "RemoveContainer" containerID="1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"
Jul 17 14:51:26.260824 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.260805    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5\": container with ID starting with 1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5 not found: ID does not exist" containerID="1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"
Jul 17 14:51:26.260901 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260827    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5"} err="failed to get container status \"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5\": rpc error: code = NotFound desc = could not find container \"1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5\": container with ID starting with 1c4d0ef174308e8ebf0c62b14c06e614cd0567fe535c19023fd803b38d3ae7c5 not found: ID does not exist"
Jul 17 14:51:26.260901 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.260840    2564 scope.go:117] "RemoveContainer" containerID="2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"
Jul 17 14:51:26.261171 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.261154    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2\": container with ID starting with 2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2 not found: ID does not exist" containerID="2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"
Jul 17 14:51:26.261216 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.261177    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2"} err="failed to get container status \"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2\": rpc error: code = NotFound desc = could not find container \"2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2\": container with ID starting with 2b0476bde691ab7b3153a531aca1d31ff8188dbad562b72ddd46411dbf63fdc2 not found: ID does not exist"
Jul 17 14:51:26.261466 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.261450    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-q9nq6"]
Jul 17 14:51:26.262216 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.262199    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 14:51:26.263102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.263084    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 14:51:26.263102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.263092    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 14:51:26.263318 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.263304    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 14:51:26.264029 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.264011    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6r5mr\""
Jul 17 14:51:26.264138 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.264083    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 14:51:26.264138 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.264108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 14:51:26.286436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286414    2564 reconciler_common.go:299] "Volume detached for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-etc-openvswitch\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.286436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286430    2564 reconciler_common.go:299] "Volume detached for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-ovn-node-metrics-cert\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.286551 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286441    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbs2l\" (UniqueName: \"kubernetes.io/projected/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-kube-api-access-pbs2l\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.286551 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286450    2564 reconciler_common.go:299] "Volume detached for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-ovn\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.286551 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286458    2564 reconciler_common.go:299] "Volume detached for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-host-run-ovn-kubernetes\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.286551 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.286467    2564 reconciler_common.go:299] "Volume detached for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4-run-systemd\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:51:26.387035 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-systemd-units\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387035 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-node-log\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5skc\" (UniqueName: \"kubernetes.io/projected/9b347ceb-531c-4b5f-a495-5320f7f01f6f-kube-api-access-k5skc\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovn-node-metrics-cert\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387203    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-ovn\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387227    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-script-lib\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-config\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387313    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-bin\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-env-overrides\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387344    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-systemd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-var-lib-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387431    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387481    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-slash\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-kubelet\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387545    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-etc-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-netns\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387586    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-log-socket\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.387716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.387601    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-netd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488305 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-script-lib\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488305 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-config\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488305 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-bin\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488316    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-env-overrides\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-systemd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488437    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-var-lib-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-systemd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-slash\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-var-lib-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-kubelet\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-slash\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488635    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-etc-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488671    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-etc-openvswitch\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-kubelet\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488403    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-bin\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-netns\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-run-netns\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-log-socket\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488778    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-netd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488807    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-systemd-units\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-node-log\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-log-socket\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488851    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5skc\" (UniqueName: \"kubernetes.io/projected/9b347ceb-531c-4b5f-a495-5320f7f01f6f-kube-api-access-k5skc\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-cni-netd\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488876    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-node-log\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488883    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-systemd-units\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.488964 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488905    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-env-overrides\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-config\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488927    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovnkube-script-lib\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovn-node-metrics-cert\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-ovn\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.488996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.489021    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-run-ovn\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.489660 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.489050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b347ceb-531c-4b5f-a495-5320f7f01f6f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.491286 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.491268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9b347ceb-531c-4b5f-a495-5320f7f01f6f-ovn-node-metrics-cert\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.500295 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.500278    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5skc\" (UniqueName: \"kubernetes.io/projected/9b347ceb-531c-4b5f-a495-5320f7f01f6f-kube-api-access-k5skc\") pod \"ovnkube-node-lk2h9\" (UID: \"9b347ceb-531c-4b5f-a495-5320f7f01f6f\") " pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.566197 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.566170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:26.576212 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:26.576191    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b347ceb_531c_4b5f_a495_5320f7f01f6f.slice/crio-6bf59b7c0fe69a9ec122e1a4f2f894790ca8827cb0007ede7825aaf2147ce68e WatchSource:0}: Error finding container 6bf59b7c0fe69a9ec122e1a4f2f894790ca8827cb0007ede7825aaf2147ce68e: Status 404 returned error can't find the container with id 6bf59b7c0fe69a9ec122e1a4f2f894790ca8827cb0007ede7825aaf2147ce68e
Jul 17 14:51:26.944425 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.944257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:26.944586 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:26.944304    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:26.944586 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.944505    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:26.944586 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:26.944564    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:27.215434 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"3dbb1c0f3e7b26446280c835680c4af5eecf6917ca17b93e470cfb00bd2dabea"}
Jul 17 14:51:27.215599 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"d75af0f99a2d4b53db216947fe6200ca37843f57deb19d457b6a2f5f34e8a816"}
Jul 17 14:51:27.215599 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"833249cdab4ea3dd1bd0277b774f554ad585f5b46af59ec70fb2eb6ca49065e5"}
Jul 17 14:51:27.215599 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"b25d7d18fdd1e5820843e8bdcc3ed25fe1caae58d778b4129c1c12a25c9c4ff7"}
Jul 17 14:51:27.215717 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215635    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"a783231082655f0a33a13230360ccbf0917885ce41f844ce3e5b3d3c0538f6fd"}
Jul 17 14:51:27.215717 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"fae807041c03af8ba3d42f6b47f7735e8a4a0a12ab295ecf46f1fd1892d7b3df"}
Jul 17 14:51:27.215717 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.215715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"6bf59b7c0fe69a9ec122e1a4f2f894790ca8827cb0007ede7825aaf2147ce68e"}
Jul 17 14:51:27.744813 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.744786    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bvflg"]
Jul 17 14:51:27.747577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.747562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:27.749741 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.749724    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 14:51:27.750021 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.750002    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 14:51:27.750590 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.750578    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m725j\""
Jul 17 14:51:27.899657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.899621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqsr\" (UniqueName: \"kubernetes.io/projected/69b566a4-7ec7-483b-b348-2a7b08dcceef-kube-api-access-pqqsr\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:27.899772 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.899672    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b566a4-7ec7-483b-b348-2a7b08dcceef-tmp-dir\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:27.899825 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.899790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b566a4-7ec7-483b-b348-2a7b08dcceef-hosts-file\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:27.944532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.944508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:27.944646 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:27.944624    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:27.948489 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:27.947766    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39cdcb6d-2d42-4b20-80d6-4fb3914a92c4" path="/var/lib/kubelet/pods/39cdcb6d-2d42-4b20-80d6-4fb3914a92c4/volumes"
Jul 17 14:51:28.000273 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.000212    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b566a4-7ec7-483b-b348-2a7b08dcceef-hosts-file\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.000273 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.000244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqsr\" (UniqueName: \"kubernetes.io/projected/69b566a4-7ec7-483b-b348-2a7b08dcceef-kube-api-access-pqqsr\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.000273 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.000270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b566a4-7ec7-483b-b348-2a7b08dcceef-tmp-dir\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.000423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.000343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/69b566a4-7ec7-483b-b348-2a7b08dcceef-hosts-file\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.000535 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.000522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69b566a4-7ec7-483b-b348-2a7b08dcceef-tmp-dir\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.011370 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.011345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqsr\" (UniqueName: \"kubernetes.io/projected/69b566a4-7ec7-483b-b348-2a7b08dcceef-kube-api-access-pqqsr\") pod \"node-resolver-bvflg\" (UID: \"69b566a4-7ec7-483b-b348-2a7b08dcceef\") " pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.056152 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.056122    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bvflg"
Jul 17 14:51:28.065395 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:28.065370    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b566a4_7ec7_483b_b348_2a7b08dcceef.slice/crio-ecd1232aa8aa56a73a691a126b6a9e9857f92526ae6dad3986fd79c4556bd86c WatchSource:0}: Error finding container ecd1232aa8aa56a73a691a126b6a9e9857f92526ae6dad3986fd79c4556bd86c: Status 404 returned error can't find the container with id ecd1232aa8aa56a73a691a126b6a9e9857f92526ae6dad3986fd79c4556bd86c
Jul 17 14:51:28.219146 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.219117    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bvflg" event={"ID":"69b566a4-7ec7-483b-b348-2a7b08dcceef","Type":"ContainerStarted","Data":"a04b6ae9898780b5f67d6163beefdf990d7c60ea3db95c2c42e5d8c4082ec435"}
Jul 17 14:51:28.219274 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.219157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bvflg" event={"ID":"69b566a4-7ec7-483b-b348-2a7b08dcceef","Type":"ContainerStarted","Data":"ecd1232aa8aa56a73a691a126b6a9e9857f92526ae6dad3986fd79c4556bd86c"}
Jul 17 14:51:28.944616 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.944595    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:28.944901 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:28.944598    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:28.944901 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:28.944685    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:28.944901 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:28.944763    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:29.224980 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:29.224892    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"ae6b6799724bdb242f1a7be98e95f688a2f7db2b407d4937841efbbf5cdbf2a4"}
Jul 17 14:51:29.945708 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:29.945678    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:29.946058 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:29.945778    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:30.944375 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:30.944333    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:30.944557 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:30.944333    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:30.944557 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:30.944443    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:30.944557 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:30.944535    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:31.232690 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.232601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" event={"ID":"9b347ceb-531c-4b5f-a495-5320f7f01f6f","Type":"ContainerStarted","Data":"ded1fd7de6bc8442df644f0dee156f82fd24f2d35d7e75bb47ef2cade930de7f"}
Jul 17 14:51:31.233148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.232843    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:31.233148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.232900    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:31.247695 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.247674    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:31.258315 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.258279    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bvflg" podStartSLOduration=4.258265234 podStartE2EDuration="4.258265234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:28.242818555 +0000 UTC m=+98.809849688" watchObservedRunningTime="2026-07-17 14:51:31.258265234 +0000 UTC m=+101.825296365"
Jul 17 14:51:31.258633 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.258614    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9" podStartSLOduration=5.258606641 podStartE2EDuration="5.258606641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:31.257976075 +0000 UTC m=+101.825007231" watchObservedRunningTime="2026-07-17 14:51:31.258606641 +0000 UTC m=+101.825638060"
Jul 17 14:51:31.944806 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:31.944771    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:31.947734 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:31.945200    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:32.235336 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:32.235256    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:32.249406 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:32.249383    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:51:32.944080 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:32.944038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:32.944250 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:32.944146    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:32.944250 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:32.944201    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:32.944369 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:32.944325    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:33.439216 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.439005    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xqvs7"]
Jul 17 14:51:33.439777 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.439333    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:33.439777 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:33.439445    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:33.439868 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.439801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6hdkd"]
Jul 17 14:51:33.439916 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.439871    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:33.439958 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:33.439943    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:33.441055 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.441034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c8d6d"]
Jul 17 14:51:33.441170 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:33.441126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:33.441229 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:33.441210    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:34.944788 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:34.944754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:34.944788 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:34.944788    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:34.945218 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:34.944869    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xqvs7" podUID="c2616f3d-01dc-4944-8fc3-13a91239d615"
Jul 17 14:51:34.945218 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:34.944903    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6hdkd" podUID="a3f0b694-edc9-4e75-81de-7b528fa2195f"
Jul 17 14:51:34.945218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:34.944945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:34.945218 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:34.945029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8d6d" podUID="ea32a2a8-657e-498b-a573-0c85ffa9eec3"
Jul 17 14:51:36.739872 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.739846    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-81.ec2.internal" event="NodeReady"
Jul 17 14:51:36.740230 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.739989    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 14:51:36.798602 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.798215    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c4d4bdd7d-ln92r"]
Jul 17 14:51:36.804013 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.803993    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"]
Jul 17 14:51:36.804141 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.804122    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.806248 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.806228    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"]
Jul 17 14:51:36.806356 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.806340    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:36.808235 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.808217    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-jlhcq"]
Jul 17 14:51:36.808347 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.808315    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"
Jul 17 14:51:36.809869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.809850    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"]
Jul 17 14:51:36.809972 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.809946    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:36.810526 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.810504    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 17 14:51:36.810620 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.810558    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 17 14:51:36.810620 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.810558    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.811883 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.811865    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"]
Jul 17 14:51:36.811987 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.811974    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:36.814459 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.814442    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ls4vc"]
Jul 17 14:51:36.814563 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.814550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:36.816655 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.816634    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"]
Jul 17 14:51:36.816813 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.816780    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.818932 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.818886    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:51:36.819416 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.819400    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:36.821034 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.821018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.828056 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.828039    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 17 14:51:36.828992 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.828966    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c4d4bdd7d-ln92r"]
Jul 17 14:51:36.829167 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.829138    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 17 14:51:36.830219 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.830204    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.830329 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.830310    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6sq5p\""
Jul 17 14:51:36.831213 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.831193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"]
Jul 17 14:51:36.879869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.879711    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 17 14:51:36.879869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.879755    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.879869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.879787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.880115 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.880047    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.880567 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.880548    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"]
Jul 17 14:51:36.881657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.881638    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.881750 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.881643    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 14:51:36.882241 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882223    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 17 14:51:36.882360 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882254    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.882456 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882289    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 17 14:51:36.882456 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.882546 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882536    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 17 14:51:36.882655 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882635    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.882748 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882728    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.882748 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.882735    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.883338 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883320    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ls4vc"]
Jul 17 14:51:36.883523 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883506    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.883603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883590    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 14:51:36.883658 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883607    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6fblh\""
Jul 17 14:51:36.883658 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883631    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.883658 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883643    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.883778 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.883593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 17 14:51:36.884501 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.884484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 14:51:36.885424 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.885411    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 14:51:36.887171 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.887152    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wbdf2\""
Jul 17 14:51:36.887266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.887178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 17 14:51:36.887266 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.887155    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.887494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.887473    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 14:51:36.887577 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.887477    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9rrxw\""
Jul 17 14:51:36.893815 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.893798    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 14:51:36.895453 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.895434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jbm86\""
Jul 17 14:51:36.895542 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.895516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 17 14:51:36.895788 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.895769    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdwrv\""
Jul 17 14:51:36.895848 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.895806    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gp27j\""
Jul 17 14:51:36.896474 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.896393    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-94tkz\""
Jul 17 14:51:36.896474 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.896455    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ptw5w\""
Jul 17 14:51:36.896648 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.896464    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 14:51:36.896852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.896837    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-jlhcq"]
Jul 17 14:51:36.900861 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.900816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:51:36.905786 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.905768    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rx4wv"]
Jul 17 14:51:36.907403 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.907385    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.908255 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.908242    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-txmxh"]
Jul 17 14:51:36.908367 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.908352    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:36.910579 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.910562    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"]
Jul 17 14:51:36.910675 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.910663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:36.912780 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.912765    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"]
Jul 17 14:51:36.912884 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.912870    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:36.914080 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914051    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 14:51:36.914228 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914114    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.914310 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914232    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 14:51:36.914310 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914246    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 14:51:36.914418 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914123    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.914674 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914661    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kbfbg\""
Jul 17 14:51:36.914835 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914817    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"]
Jul 17 14:51:36.914947 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.914930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:36.917176 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.917161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"]
Jul 17 14:51:36.917270 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.917195    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"]
Jul 17 14:51:36.917270 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.917218    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"]
Jul 17 14:51:36.917270 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.917251    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 14:51:36.917419 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.917277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:36.920018 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.920003    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"]
Jul 17 14:51:36.920134 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.920121    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"
Jul 17 14:51:36.922571 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"]
Jul 17 14:51:36.922666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-txmxh"]
Jul 17 14:51:36.922666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"]
Jul 17 14:51:36.922666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922623    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rx4wv"]
Jul 17 14:51:36.922666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922640    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"]
Jul 17 14:51:36.922666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:36.922853 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.922658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"]
Jul 17 14:51:36.923765 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.923574    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 14:51:36.927784 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.927766    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2nqmv\""
Jul 17 14:51:36.927890 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.927872    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hxssw\""
Jul 17 14:51:36.927977 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.927805    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 14:51:36.928035 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.928001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.928662 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.928645    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"]
Jul 17 14:51:36.930138 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.930116    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.930347 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.930116    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 14:51:36.930347 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.930315    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 17 14:51:36.930508 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.930482    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 17 14:51:36.931794 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.931754    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 14:51:36.931979 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.931961    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 17 14:51:36.932110 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.932012    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 14:51:36.932110 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.932012    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 17 14:51:36.932110 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.932082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qwrb9\""
Jul 17 14:51:36.932310 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.932176    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-knkk8\""
Jul 17 14:51:36.932388 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.932371    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 17 14:51:36.933439 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.933423    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 14:51:36.936844 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.936823    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"]
Jul 17 14:51:36.937785 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.937760    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 14:51:36.939642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.939620    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 14:51:36.944292 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.944259    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:36.944292 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.944281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:51:36.944433 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.944282    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:36.947283 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.947268    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 14:51:36.947383 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.947300    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k8rtv\""
Jul 17 14:51:36.948029 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.948015    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7594c\""
Jul 17 14:51:36.948197 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.948181    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 14:51:36.966622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttjbt\" (UniqueName: \"kubernetes.io/projected/26575325-fed3-4634-8f77-278b4bd35d45-kube-api-access-ttjbt\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.966705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966627    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.966705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gh8v\" (UniqueName: \"kubernetes.io/projected/faed7fae-93eb-4f34-829c-af921104356d-kube-api-access-8gh8v\") pod \"volume-data-source-validator-78cf4c6f6-xv295\" (UID: \"faed7fae-93eb-4f34-829c-af921104356d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"
Jul 17 14:51:36.966705 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5n4c\" (UniqueName: \"kubernetes.io/projected/49279156-70fd-4c62-b3e3-2f02b820eecb-kube-api-access-v5n4c\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:36.966798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966722    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.966798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966757    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26575325-fed3-4634-8f77-278b4bd35d45-serving-cert\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.966798 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5q44\" (UniqueName: \"kubernetes.io/projected/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-kube-api-access-c5q44\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:36.966884 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966797    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-serving-cert\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:36.966884 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966833    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-tmp\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.966884 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.966968 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966890    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8126ee2-942d-4fea-a824-2e251bcf23b0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:36.966968 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xfx\" (UniqueName: \"kubernetes.io/projected/f8126ee2-942d-4fea-a824-2e251bcf23b0-kube-api-access-c9xfx\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:36.966968 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.966968 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966953    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-trusted-ca\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:36.967102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.966986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.967102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967008    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:36.967102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967033    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjs8j\" (UniqueName: \"kubernetes.io/projected/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-kube-api-access-rjs8j\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:36.967102 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-default-certificate\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0baed11-a685-4de7-b51d-2316e2feba8b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967177    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0baed11-a685-4de7-b51d-2316e2feba8b-config\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:36.967218 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-stats-auth\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85z6f\" (UniqueName: \"kubernetes.io/projected/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-kube-api-access-85z6f\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967256    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-config\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-snapshots\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-service-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.967387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvsf\" (UniqueName: \"kubernetes.io/projected/a0baed11-a685-4de7-b51d-2316e2feba8b-kube-api-access-qxvsf\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:36.967578 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967418    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:36.967578 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:36.967578 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967476    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5wwq\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:36.967578 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:36.967493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.068666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.068666 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26575325-fed3-4634-8f77-278b4bd35d45-serving-cert\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5q44\" (UniqueName: \"kubernetes.io/projected/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-kube-api-access-c5q44\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83176294-865b-44d2-81f8-a25dd8b6ff38-tmp\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-serving-cert\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-tmp\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjx5\" (UniqueName: \"kubernetes.io/projected/76714e7f-b424-4427-8d5b-b1f681dd35b0-kube-api-access-hjjx5\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.068888 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8126ee2-942d-4fea-a824-2e251bcf23b0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xfx\" (UniqueName: \"kubernetes.io/projected/f8126ee2-942d-4fea-a824-2e251bcf23b0-kube-api-access-c9xfx\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.068985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-trusted-ca\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069010    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-tmp-dir\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069104    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjs8j\" (UniqueName: \"kubernetes.io/projected/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-kube-api-access-rjs8j\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-default-certificate\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.069204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069248    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-tmp\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069273    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc284a53-0080-4ac6-9e8a-a46e170d7a87-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.069642 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069355    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0baed11-a685-4de7-b51d-2316e2feba8b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.069876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.069876 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.069680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f8126ee2-942d-4fea-a824-2e251bcf23b0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.070136 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.070113    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.070282 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.070145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-trusted-ca\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.070410 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.069184    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:37.070668 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.069389    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.569371101 +0000 UTC m=+108.136402231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:37.070813 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.070795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0baed11-a685-4de7-b51d-2316e2feba8b-config\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.070167    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a0baed11-a685-4de7-b51d-2316e2feba8b-config\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.071004    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.570979682 +0000 UTC m=+108.138010800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.071481    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.571462567 +0000 UTC m=+108.138493689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls882\" (UniqueName: \"kubernetes.io/projected/a306a5b7-21a5-4135-9294-e48b5fb6793a-kube-api-access-ls882\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071555    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-stats-auth\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85z6f\" (UniqueName: \"kubernetes.io/projected/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-kube-api-access-85z6f\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071640    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071716    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-config-volume\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071743    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-config\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-snapshots\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.072635 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/83176294-865b-44d2-81f8-a25dd8b6ff38-klusterlet-config\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071888    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-service-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.072173    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.072187    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.072239    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.57222533 +0000 UTC m=+108.139256448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.071919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvsf\" (UniqueName: \"kubernetes.io/projected/a0baed11-a685-4de7-b51d-2316e2feba8b-kube-api-access-qxvsf\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjh5c\" (UniqueName: \"kubernetes.io/projected/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-kube-api-access-xjh5c\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-service-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjcdt\" (UniqueName: \"kubernetes.io/projected/83176294-865b-44d2-81f8-a25dd8b6ff38-kube-api-access-wjcdt\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a306a5b7-21a5-4135-9294-e48b5fb6793a-cert\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-config\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e99fae97-dd94-4839-8a37-61ddccaa902b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072918    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jwt\" (UniqueName: \"kubernetes.io/projected/e99fae97-dd94-4839-8a37-61ddccaa902b-kube-api-access-f2jwt\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.073852 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26575325-fed3-4634-8f77-278b4bd35d45-snapshots\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.072996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48vrd\" (UniqueName: \"kubernetes.io/projected/0287a1b6-c764-453f-9c88-84a5ad805084-kube-api-access-48vrd\") pod \"network-check-source-7bc5b6bf85-td64b\" (UID: \"0287a1b6-c764-453f-9c88-84a5ad805084\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5wwq\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttjbt\" (UniqueName: \"kubernetes.io/projected/26575325-fed3-4634-8f77-278b4bd35d45-kube-api-access-ttjbt\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gh8v\" (UniqueName: \"kubernetes.io/projected/faed7fae-93eb-4f34-829c-af921104356d-kube-api-access-8gh8v\") pod \"volume-data-source-validator-78cf4c6f6-xv295\" (UID: \"faed7fae-93eb-4f34-829c-af921104356d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a0baed11-a685-4de7-b51d-2316e2feba8b-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073261    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e7f-b424-4427-8d5b-b1f681dd35b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5n4c\" (UniqueName: \"kubernetes.io/projected/49279156-70fd-4c62-b3e3-2f02b820eecb-kube-api-access-v5n4c\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073325    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26575325-fed3-4634-8f77-278b4bd35d45-serving-cert\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.073447    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.073500    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.573486661 +0000 UTC m=+108.140517771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.074937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073541    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-default-certificate\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.075582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073595    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-serving-cert\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.075582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26575325-fed3-4634-8f77-278b4bd35d45-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.075582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.073832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.075582 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.074002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.076056 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.076036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.076420 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.076399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-stats-auth\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.078149 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.078129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5q44\" (UniqueName: \"kubernetes.io/projected/ddc5821f-f2b8-4072-9b11-abf6a3ac74ac-kube-api-access-c5q44\") pod \"console-operator-565b7b569d-jlhcq\" (UID: \"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac\") " pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.079985 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.079965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xfx\" (UniqueName: \"kubernetes.io/projected/f8126ee2-942d-4fea-a824-2e251bcf23b0-kube-api-access-c9xfx\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.080275 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.080254    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjs8j\" (UniqueName: \"kubernetes.io/projected/a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c-kube-api-access-rjs8j\") pod \"kube-storage-version-migrator-operator-7658d9b749-7j5w7\" (UID: \"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.080560 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.080537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85z6f\" (UniqueName: \"kubernetes.io/projected/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-kube-api-access-85z6f\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.081339 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.081296    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.083637 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.083613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5n4c\" (UniqueName: \"kubernetes.io/projected/49279156-70fd-4c62-b3e3-2f02b820eecb-kube-api-access-v5n4c\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:37.083749 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.083658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gh8v\" (UniqueName: \"kubernetes.io/projected/faed7fae-93eb-4f34-829c-af921104356d-kube-api-access-8gh8v\") pod \"volume-data-source-validator-78cf4c6f6-xv295\" (UID: \"faed7fae-93eb-4f34-829c-af921104356d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"
Jul 17 14:51:37.083749 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.083731    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvsf\" (UniqueName: \"kubernetes.io/projected/a0baed11-a685-4de7-b51d-2316e2feba8b-kube-api-access-qxvsf\") pod \"service-ca-operator-5bcbc94c78-nk975\" (UID: \"a0baed11-a685-4de7-b51d-2316e2feba8b\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.084017 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.083999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5wwq\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.084130 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.084047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttjbt\" (UniqueName: \"kubernetes.io/projected/26575325-fed3-4634-8f77-278b4bd35d45-kube-api-access-ttjbt\") pod \"insights-operator-555d9d7885-ls4vc\" (UID: \"26575325-fed3-4634-8f77-278b4bd35d45\") " pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.127342 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.127279    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"
Jul 17 14:51:37.133392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.133372    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:37.139917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.139890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"
Jul 17 14:51:37.152712 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.152689    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ls4vc"
Jul 17 14:51:37.159019 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.158993    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"
Jul 17 14:51:37.174914 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.174877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-config-volume\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.175032 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.174930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.175032 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.174967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/83176294-865b-44d2-81f8-a25dd8b6ff38-klusterlet-config\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.175032 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjh5c\" (UniqueName: \"kubernetes.io/projected/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-kube-api-access-xjh5c\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjcdt\" (UniqueName: \"kubernetes.io/projected/83176294-865b-44d2-81f8-a25dd8b6ff38-kube-api-access-wjcdt\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a306a5b7-21a5-4135-9294-e48b5fb6793a-cert\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e99fae97-dd94-4839-8a37-61ddccaa902b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jwt\" (UniqueName: \"kubernetes.io/projected/e99fae97-dd94-4839-8a37-61ddccaa902b-kube-api-access-f2jwt\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175222 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48vrd\" (UniqueName: \"kubernetes.io/projected/0287a1b6-c764-453f-9c88-84a5ad805084-kube-api-access-48vrd\") pod \"network-check-source-7bc5b6bf85-td64b\" (UID: \"0287a1b6-c764-453f-9c88-84a5ad805084\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e7f-b424-4427-8d5b-b1f681dd35b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83176294-865b-44d2-81f8-a25dd8b6ff38-tmp\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjx5\" (UniqueName: \"kubernetes.io/projected/76714e7f-b424-4427-8d5b-b1f681dd35b0-kube-api-access-hjjx5\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-tmp-dir\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-config-volume\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.175572 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175554    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.175930 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.175930 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc284a53-0080-4ac6-9e8a-a46e170d7a87-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.175930 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.175689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls882\" (UniqueName: \"kubernetes.io/projected/a306a5b7-21a5-4135-9294-e48b5fb6793a-kube-api-access-ls882\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.178351 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.176719    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:37.178351 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.176790    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.676772023 +0000 UTC m=+108.243803146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:37.178351 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.177672    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bc284a53-0080-4ac6-9e8a-a46e170d7a87-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.178351 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.178268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/76714e7f-b424-4427-8d5b-b1f681dd35b0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.179503 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.179043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83176294-865b-44d2-81f8-a25dd8b6ff38-tmp\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.179503 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.179156    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:37.179503 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.179220    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:37.679201987 +0000 UTC m=+108.246233099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:37.179503 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.179326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-tmp-dir\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.181956 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.181916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.183287 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.183164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-hub\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.183287 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.183176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e99fae97-dd94-4839-8a37-61ddccaa902b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.183927 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.183861    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.185146 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.184397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/76714e7f-b424-4427-8d5b-b1f681dd35b0-ca\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.185146 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.185103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/83176294-865b-44d2-81f8-a25dd8b6ff38-klusterlet-config\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.186052 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.186030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a306a5b7-21a5-4135-9294-e48b5fb6793a-cert\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.188409 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.188210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls882\" (UniqueName: \"kubernetes.io/projected/a306a5b7-21a5-4135-9294-e48b5fb6793a-kube-api-access-ls882\") pod \"ingress-canary-rx4wv\" (UID: \"a306a5b7-21a5-4135-9294-e48b5fb6793a\") " pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.191126 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.191045    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjh5c\" (UniqueName: \"kubernetes.io/projected/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-kube-api-access-xjh5c\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.191716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.191690    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jwt\" (UniqueName: \"kubernetes.io/projected/e99fae97-dd94-4839-8a37-61ddccaa902b-kube-api-access-f2jwt\") pod \"managed-serviceaccount-addon-agent-565f48c786-fvpqq\" (UID: \"e99fae97-dd94-4839-8a37-61ddccaa902b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.191806 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.191758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48vrd\" (UniqueName: \"kubernetes.io/projected/0287a1b6-c764-453f-9c88-84a5ad805084-kube-api-access-48vrd\") pod \"network-check-source-7bc5b6bf85-td64b\" (UID: \"0287a1b6-c764-453f-9c88-84a5ad805084\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"
Jul 17 14:51:37.192212 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.192193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjx5\" (UniqueName: \"kubernetes.io/projected/76714e7f-b424-4427-8d5b-b1f681dd35b0-kube-api-access-hjjx5\") pod \"cluster-proxy-proxy-agent-7bc9787d9b-b4bzn\" (UID: \"76714e7f-b424-4427-8d5b-b1f681dd35b0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.196860 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.196832    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjcdt\" (UniqueName: \"kubernetes.io/projected/83176294-865b-44d2-81f8-a25dd8b6ff38-kube-api-access-wjcdt\") pod \"klusterlet-addon-workmgr-868dbcc5bb-79rz4\" (UID: \"83176294-865b-44d2-81f8-a25dd8b6ff38\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.220467 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.220196    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rx4wv"
Jul 17 14:51:37.240027 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.239816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"
Jul 17 14:51:37.254604 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.254377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"
Jul 17 14:51:37.261524 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.261298    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:51:37.296143 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.290288    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:37.418363 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.418329    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295"]
Jul 17 14:51:37.422341 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.421745    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7"]
Jul 17 14:51:37.434679 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.433778    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975"]
Jul 17 14:51:37.450847 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.444976    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0baed11_a685_4de7_b51d_2316e2feba8b.slice/crio-1add200cd1b5b38751f6e091b96baf8d3ac293ee7d4177fd94de651e0a913b36 WatchSource:0}: Error finding container 1add200cd1b5b38751f6e091b96baf8d3ac293ee7d4177fd94de651e0a913b36: Status 404 returned error can't find the container with id 1add200cd1b5b38751f6e091b96baf8d3ac293ee7d4177fd94de651e0a913b36
Jul 17 14:51:37.458168 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.458058    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ls4vc"]
Jul 17 14:51:37.469710 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.461499    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-jlhcq"]
Jul 17 14:51:37.514772 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.514716    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rx4wv"]
Jul 17 14:51:37.519603 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.519573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda306a5b7_21a5_4135_9294_e48b5fb6793a.slice/crio-46e4dabff9f32453b354eef93ad0ca75934158fa0de248781bef403155167bd8 WatchSource:0}: Error finding container 46e4dabff9f32453b354eef93ad0ca75934158fa0de248781bef403155167bd8: Status 404 returned error can't find the container with id 46e4dabff9f32453b354eef93ad0ca75934158fa0de248781bef403155167bd8
Jul 17 14:51:37.521158 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.521136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b"]
Jul 17 14:51:37.524837 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.524814    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0287a1b6_c764_453f_9c88_84a5ad805084.slice/crio-0a14fe1039a02d227778efec1f207a6810cbeec8c7c34916fdc7545b3809b195 WatchSource:0}: Error finding container 0a14fe1039a02d227778efec1f207a6810cbeec8c7c34916fdc7545b3809b195: Status 404 returned error can't find the container with id 0a14fe1039a02d227778efec1f207a6810cbeec8c7c34916fdc7545b3809b195
Jul 17 14:51:37.532597 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.532575    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq"]
Jul 17 14:51:37.535277 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.535250    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode99fae97_dd94_4839_8a37_61ddccaa902b.slice/crio-da83386872ce8c0b81eb69c19d486fd0c1a7fe9717143206416ff7e509d25aa5 WatchSource:0}: Error finding container da83386872ce8c0b81eb69c19d486fd0c1a7fe9717143206416ff7e509d25aa5: Status 404 returned error can't find the container with id da83386872ce8c0b81eb69c19d486fd0c1a7fe9717143206416ff7e509d25aa5
Jul 17 14:51:37.535632 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.535610    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"]
Jul 17 14:51:37.538030 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.538009    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76714e7f_b424_4427_8d5b_b1f681dd35b0.slice/crio-8950477c2db1e01a4cad3fca706d139857734f877e444fffffde092bf1848b17 WatchSource:0}: Error finding container 8950477c2db1e01a4cad3fca706d139857734f877e444fffffde092bf1848b17: Status 404 returned error can't find the container with id 8950477c2db1e01a4cad3fca706d139857734f877e444fffffde092bf1848b17
Jul 17 14:51:37.543136 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.543117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"]
Jul 17 14:51:37.545538 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:37.545519    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83176294_865b_44d2_81f8_a25dd8b6ff38.slice/crio-42f06745d0f0843b137a2e819be4bd28ae57c80c95a475f4be720406cb3ef8bc WatchSource:0}: Error finding container 42f06745d0f0843b137a2e819be4bd28ae57c80c95a475f4be720406cb3ef8bc: Status 404 returned error can't find the container with id 42f06745d0f0843b137a2e819be4bd28ae57c80c95a475f4be720406cb3ef8bc
Jul 17 14:51:37.580352 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.580327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:37.580475 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.580390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.580475 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.580420    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:37.580475 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.580441    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:37.580475 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.580472    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580499    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580572    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.580550587 +0000 UTC m=+109.147581711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580580    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580595    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.580584905 +0000 UTC m=+109.147616021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580581    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580632    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.580615317 +0000 UTC m=+109.147646426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580585    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580653    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:37.580673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580660    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.580646191 +0000 UTC m=+109.147677304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:37.580968 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.580683    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.58067362 +0000 UTC m=+109.147704763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:37.681498 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.681398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:37.681498 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.681481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:37.681673 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.681519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:37.681673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.681623    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:37.681673 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.681633    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:37.681795 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.681690    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.681671077 +0000 UTC m=+109.248702205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:37.681795 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:37.681708    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:38.681699978 +0000 UTC m=+109.248731087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:37.684480 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.684457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b869d\" (UniqueName: \"kubernetes.io/projected/c2616f3d-01dc-4944-8fc3-13a91239d615-kube-api-access-b869d\") pod \"network-check-target-xqvs7\" (UID: \"c2616f3d-01dc-4944-8fc3-13a91239d615\") " pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:37.895825 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:37.895795    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:38.057235 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.057201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xqvs7"]
Jul 17 14:51:38.064240 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:38.063536    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2616f3d_01dc_4944_8fc3_13a91239d615.slice/crio-676607c8f145eebee2273f94dbebc1ac8b26e5ce59cf6ac097c4173d9d355d20 WatchSource:0}: Error finding container 676607c8f145eebee2273f94dbebc1ac8b26e5ce59cf6ac097c4173d9d355d20: Status 404 returned error can't find the container with id 676607c8f145eebee2273f94dbebc1ac8b26e5ce59cf6ac097c4173d9d355d20
Jul 17 14:51:38.255087 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.254948    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq" event={"ID":"e99fae97-dd94-4839-8a37-61ddccaa902b","Type":"ContainerStarted","Data":"da83386872ce8c0b81eb69c19d486fd0c1a7fe9717143206416ff7e509d25aa5"}
Jul 17 14:51:38.256449 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.256421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b" event={"ID":"0287a1b6-c764-453f-9c88-84a5ad805084","Type":"ContainerStarted","Data":"0a14fe1039a02d227778efec1f207a6810cbeec8c7c34916fdc7545b3809b195"}
Jul 17 14:51:38.258059 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.258005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerStarted","Data":"8950477c2db1e01a4cad3fca706d139857734f877e444fffffde092bf1848b17"}
Jul 17 14:51:38.260639 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.260567    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295" event={"ID":"faed7fae-93eb-4f34-829c-af921104356d","Type":"ContainerStarted","Data":"4d092c980e3dd1c3e14f1231cf08cf70858f1341cf2c75c6bd785dfc217abcb9"}
Jul 17 14:51:38.262920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.262853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xqvs7" event={"ID":"c2616f3d-01dc-4944-8fc3-13a91239d615","Type":"ContainerStarted","Data":"676607c8f145eebee2273f94dbebc1ac8b26e5ce59cf6ac097c4173d9d355d20"}
Jul 17 14:51:38.269052 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.268990    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4" event={"ID":"83176294-865b-44d2-81f8-a25dd8b6ff38","Type":"ContainerStarted","Data":"42f06745d0f0843b137a2e819be4bd28ae57c80c95a475f4be720406cb3ef8bc"}
Jul 17 14:51:38.272589 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.272517    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rx4wv" event={"ID":"a306a5b7-21a5-4135-9294-e48b5fb6793a","Type":"ContainerStarted","Data":"46e4dabff9f32453b354eef93ad0ca75934158fa0de248781bef403155167bd8"}
Jul 17 14:51:38.275240 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.275214    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975" event={"ID":"a0baed11-a685-4de7-b51d-2316e2feba8b","Type":"ContainerStarted","Data":"1add200cd1b5b38751f6e091b96baf8d3ac293ee7d4177fd94de651e0a913b36"}
Jul 17 14:51:38.281051 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.280998    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ls4vc" event={"ID":"26575325-fed3-4634-8f77-278b4bd35d45","Type":"ContainerStarted","Data":"a1fc07eb6e19d4c78916de10293aaa21922ccf81c4c781d34b122672ba4796b1"}
Jul 17 14:51:38.283526 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.283465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7" event={"ID":"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c","Type":"ContainerStarted","Data":"51e6b545961e5fbac312d536c49f6376f9b52861629750ac8b450e75db1c5af0"}
Jul 17 14:51:38.286345 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.286318    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" event={"ID":"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac","Type":"ContainerStarted","Data":"6f664d1e3cf4520cdd8540751a6574a9f6f5b5b34de05e4554052aa1b0ebdd75"}
Jul 17 14:51:38.589611 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.589577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:38.589823 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.589629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:38.589823 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.589661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:38.589823 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.589708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:38.589823 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.589776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:38.590038 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.589902    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:38.590038 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.589964    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.589944378 +0000 UTC m=+111.156975505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590265    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590294    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590316    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590323    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.590307985 +0000 UTC m=+111.157339102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590370    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.590352111 +0000 UTC m=+111.157383224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590382    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590416    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.590405226 +0000 UTC m=+111.157436336 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:38.590477 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.590431    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.590423023 +0000 UTC m=+111.157454134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.697272    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:38.697359    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.697686    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.697809    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.697787717 +0000 UTC m=+111.264818828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.698410    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:38.698511 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:38.698466    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:40.698443436 +0000 UTC m=+111.265474554 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.618734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.618795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.618826    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.618877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.618951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619152    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619221    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.619201031 +0000 UTC m=+115.186232160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619626    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619676    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.619659156 +0000 UTC m=+115.186690265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619741    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.619730797 +0000 UTC m=+115.186761911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619799    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619830    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.619819241 +0000 UTC m=+115.186850356 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619889    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619902    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:40.620031 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.619939    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.61992932 +0000 UTC m=+115.186960435 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:40.719677 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.719641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:40.719989 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:40.719720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:40.719989 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.719957    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:40.720260 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.720020    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.720001851 +0000 UTC m=+115.287032978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:40.720469 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.720428    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:40.720539 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:40.720484    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:44.720468031 +0000 UTC m=+115.287499151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:41.332997 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:41.332961    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295" event={"ID":"faed7fae-93eb-4f34-829c-af921104356d","Type":"ContainerStarted","Data":"ddd7c542269f86a108729028d5679c4ddf1303c405eadc0504a8330a1b0950e9"}
Jul 17 14:51:44.662795 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.662720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:44.662795 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.662782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.662815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.662865    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.662927    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.662946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.662967    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.662949656 +0000 UTC m=+123.229980767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.662994    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663022    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.663000663 +0000 UTC m=+123.230031782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663034    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663036    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663046    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663093    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.663057591 +0000 UTC m=+123.230088714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663150    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.663137745 +0000 UTC m=+123.230168869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:44.663436 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.663169    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.663158945 +0000 UTC m=+123.230190057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:44.763999 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.763958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:44.764204 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.764110    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:44.764204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:44.764183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:44.764295 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.764265    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.764236105 +0000 UTC m=+123.331267219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:44.764335 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.764297    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:44.764365 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:44.764349    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.764333388 +0000 UTC m=+123.331364503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:49.362871 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.362840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xqvs7" event={"ID":"c2616f3d-01dc-4944-8fc3-13a91239d615","Type":"ContainerStarted","Data":"4b0810db4b72318f4c34aaa514e6ff52ebef47a3bf5399a1e4b12c4d7df8d73f"}
Jul 17 14:51:49.363275 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.363133    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:51:49.364904 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.364838    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4" event={"ID":"83176294-865b-44d2-81f8-a25dd8b6ff38","Type":"ContainerStarted","Data":"7fd8e9bb46bc3fb03f737766c0c1841c542855f015dde8d52dfb526e5b7057ba"}
Jul 17 14:51:49.365214 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.365172    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:49.366928 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.366885    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4"
Jul 17 14:51:49.369258 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.368945    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rx4wv" event={"ID":"a306a5b7-21a5-4135-9294-e48b5fb6793a","Type":"ContainerStarted","Data":"1d8a52c6cf20447dc8a90e6a0b6685306fe0e3b7408465fa2725465fff4ca57f"}
Jul 17 14:51:49.371014 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.370991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975" event={"ID":"a0baed11-a685-4de7-b51d-2316e2feba8b","Type":"ContainerStarted","Data":"ae2e57e3795e0c5cb23722f3d83ab8144fdd142a73e8e5ae923f2a03db9615da"}
Jul 17 14:51:49.373183 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.373142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ls4vc" event={"ID":"26575325-fed3-4634-8f77-278b4bd35d45","Type":"ContainerStarted","Data":"df0df311bf244f628682bae88eb712e333ed3d4d8e9ae119651c3bbcf1348db8"}
Jul 17 14:51:49.375174 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.375123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7" event={"ID":"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c","Type":"ContainerStarted","Data":"042734cb360959fb9f7a1e123772135211c469ac0b7d89b975d442556bf35c79"}
Jul 17 14:51:49.377164 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.377144    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/0.log"
Jul 17 14:51:49.377262 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.377177    2564 generic.go:356] "Generic (PLEG): container finished" podID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac" containerID="624853897eed9a18365d14ccb3a859dfbaf16fa7d15441426c57151b99837dc6" exitCode=255
Jul 17 14:51:49.377262 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.377249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" event={"ID":"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac","Type":"ContainerDied","Data":"624853897eed9a18365d14ccb3a859dfbaf16fa7d15441426c57151b99837dc6"}
Jul 17 14:51:49.377770 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.377494    2564 scope.go:117] "RemoveContainer" containerID="624853897eed9a18365d14ccb3a859dfbaf16fa7d15441426c57151b99837dc6"
Jul 17 14:51:49.381913 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.381872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq" event={"ID":"e99fae97-dd94-4839-8a37-61ddccaa902b","Type":"ContainerStarted","Data":"0735aa48c9e290af88d8554e9451fe564566fdf7f6967a2da6cccef0f76506c8"}
Jul 17 14:51:49.385657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.385602    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-xv295" podStartSLOduration=128.044406547 podStartE2EDuration="2m11.385586264s" totalImagesPullingTime="3.341179717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:41.353409681 +0000 UTC m=+111.920440814" watchObservedRunningTime="2026-07-17 14:51:49.385586264 +0000 UTC m=+119.952617397"
Jul 17 14:51:49.386354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.385965    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b" event={"ID":"0287a1b6-c764-453f-9c88-84a5ad805084","Type":"ContainerStarted","Data":"39f1ef0015fbe6d07a4fb7e77cdd379a86711ea0a23ebf17ac1c978b525c2cbf"}
Jul 17 14:51:49.386354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.386303    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xqvs7" podStartSLOduration=65.427926616 podStartE2EDuration="1m16.38629281s" totalImagesPullingTime="10.958366194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:50:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.380314401 +0000 UTC m=+119.947345534" watchObservedRunningTime="2026-07-17 14:51:49.38629281 +0000 UTC m=+119.953323942"
Jul 17 14:51:49.389262 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.389244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerStarted","Data":"0b8594aaebdd126fed2ee300e419a930f66b92afb7b150e8639dce9839c41963"}
Jul 17 14:51:49.389377 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.389365    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerStarted","Data":"cd1e56f874c31c1d4d2596f50d13c8b3e741a25cc65b523284ab861db6f1164a"}
Jul 17 14:51:49.401543 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.401500    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-868dbcc5bb-79rz4" podStartSLOduration=18.941967996 podStartE2EDuration="30.401467448s" totalImagesPullingTime="11.459499452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.399225155 +0000 UTC m=+119.966256291" watchObservedRunningTime="2026-07-17 14:51:49.401467448 +0000 UTC m=+119.968498561"
Jul 17 14:51:49.449880 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.449822    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975" podStartSLOduration=119.898516356 podStartE2EDuration="2m11.449801848s" totalImagesPullingTime="11.551285492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.449716524 +0000 UTC m=+120.016747648" watchObservedRunningTime="2026-07-17 14:51:49.449801848 +0000 UTC m=+120.016832981"
Jul 17 14:51:49.467547 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.466249    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rx4wv" podStartSLOduration=1.9813761479999998 podStartE2EDuration="13.466226958s" totalImagesPullingTime="11.48485081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.465798061 +0000 UTC m=+120.032829211" watchObservedRunningTime="2026-07-17 14:51:49.466226958 +0000 UTC m=+120.033258093"
Jul 17 14:51:49.502172 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.502058    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7" podStartSLOduration=119.956970253 podStartE2EDuration="2m11.502037408s" totalImagesPullingTime="11.545067155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.483645788 +0000 UTC m=+120.050676922" watchObservedRunningTime="2026-07-17 14:51:49.502037408 +0000 UTC m=+120.069068541"
Jul 17 14:51:49.519322 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.518778    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-ls4vc" podStartSLOduration=119.992881176 podStartE2EDuration="2m11.518758291s" totalImagesPullingTime="11.525877115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.502433742 +0000 UTC m=+120.069464871" watchObservedRunningTime="2026-07-17 14:51:49.518758291 +0000 UTC m=+120.085789423"
Jul 17 14:51:49.520195 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.520117    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-td64b" podStartSLOduration=66.036069013 podStartE2EDuration="1m17.520103254s" totalImagesPullingTime="11.484034241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.517298728 +0000 UTC m=+120.084329863" watchObservedRunningTime="2026-07-17 14:51:49.520103254 +0000 UTC m=+120.087134385"
Jul 17 14:51:49.537166 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.537114    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-565f48c786-fvpqq" podStartSLOduration=19.058539877 podStartE2EDuration="30.537095435s" totalImagesPullingTime="11.478555558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.535432271 +0000 UTC m=+120.102463404" watchObservedRunningTime="2026-07-17 14:51:49.537095435 +0000 UTC m=+120.104126563"
Jul 17 14:51:49.559296 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:49.559247    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" podStartSLOduration=19.065462018 podStartE2EDuration="30.559235452s" totalImagesPullingTime="11.493773434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:49.557905814 +0000 UTC m=+120.124936947" watchObservedRunningTime="2026-07-17 14:51:49.559235452 +0000 UTC m=+120.126266585"
Jul 17 14:51:50.395872 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.395815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerStarted","Data":"d17ae4367c8745eb3af381ae38c18729d6aa0d62a5ad9593a06dd9bbc188eb82"}
Jul 17 14:51:50.397570 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.397549    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:51:50.397997 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.397981    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/0.log"
Jul 17 14:51:50.398058 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.398020    2564 generic.go:356] "Generic (PLEG): container finished" podID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac" containerID="1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4" exitCode=255
Jul 17 14:51:50.398131 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.398104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" event={"ID":"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac","Type":"ContainerDied","Data":"1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4"}
Jul 17 14:51:50.398196 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.398152    2564 scope.go:117] "RemoveContainer" containerID="624853897eed9a18365d14ccb3a859dfbaf16fa7d15441426c57151b99837dc6"
Jul 17 14:51:50.398419 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:50.398399    2564 scope.go:117] "RemoveContainer" containerID="1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4"
Jul 17 14:51:50.398602 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:50.398573    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-jlhcq_openshift-console-operator(ddc5821f-f2b8-4072-9b11-abf6a3ac74ac)\"" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" podUID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac"
Jul 17 14:51:51.047290 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.047253    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pcnmv"]
Jul 17 14:51:51.051615 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.051593    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.054110 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.054082    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 14:51:51.054216 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.054192    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rpvrl\""
Jul 17 14:51:51.054465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.054449    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 14:51:51.069653 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.069634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pcnmv"]
Jul 17 14:51:51.129663 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.129630    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.129799 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.129701    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-crio-socket\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.129799 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.129747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-data-volume\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.129900 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.129800    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjl2l\" (UniqueName: \"kubernetes.io/projected/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-api-access-xjl2l\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.129900 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.129827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231044 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231198 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231089    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-crio-socket\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231198 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231145    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-data-volume\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231198 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.231177    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:51.231198 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231193    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjl2l\" (UniqueName: \"kubernetes.io/projected/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-api-access-xjl2l\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231375 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-crio-socket\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231375 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.231242    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:51.73122553 +0000 UTC m=+122.298256644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:51.231375 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231260    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.231375 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.231339    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 14:51:51.231571 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.231393    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:51.731374236 +0000 UTC m=+122.298405361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : configmap "kube-rbac-proxy" not found
Jul 17 14:51:51.231571 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.231507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-data-volume\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.241006 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.240983    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjl2l\" (UniqueName: \"kubernetes.io/projected/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-api-access-xjl2l\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.402514 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.402489    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:51:51.402968 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.402825    2564 scope.go:117] "RemoveContainer" containerID="1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4"
Jul 17 14:51:51.403033 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.402986    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-jlhcq_openshift-console-operator(ddc5821f-f2b8-4072-9b11-abf6a3ac74ac)\"" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" podUID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac"
Jul 17 14:51:51.737170 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.737086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.737308 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.737194    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 14:51:51.737308 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:51.737211    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:51.737308 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.737267    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.737250654 +0000 UTC m=+123.304281763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : configmap "kube-rbac-proxy" not found
Jul 17 14:51:51.737426 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.737322    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:51.737426 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:51.737369    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:52.737357214 +0000 UTC m=+123.304388329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:52.746244 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746206    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746332    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746387    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746393    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:54.746378772 +0000 UTC m=+125.313409881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : configmap "kube-rbac-proxy" not found
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746529    2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746552    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746585    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls podName:f8126ee2-942d-4fea-a824-2e251bcf23b0 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.746565962 +0000 UTC m=+139.313597079 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l5xqp" (UID: "f8126ee2-942d-4fea-a824-2e251bcf23b0") : secret "cluster-monitoring-operator-tls" not found
Jul 17 14:51:52.746605 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746611    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.746599929 +0000 UTC m=+139.313631042 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : secret "router-metrics-certs-default" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746641    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746646    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle podName:f5eb9ce6-f7ca-454c-900b-76b4cb6cae20 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.746633275 +0000 UTC m=+139.313664388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle") pod "router-default-7c4d4bdd7d-ln92r" (UID: "f5eb9ce6-f7ca-454c-900b-76b4cb6cae20") : configmap references non-existent config key: service-ca.crt
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746737    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls podName:49279156-70fd-4c62-b3e3-2f02b820eecb nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.74671745 +0000 UTC m=+139.313748564 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-sg2zs" (UID: "49279156-70fd-4c62-b3e3-2f02b820eecb") : secret "samples-operator-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746756    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746767    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-546f954549-2sb84: secret "image-registry-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.746793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746822    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746833    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls podName:c5ad06c7-71b9-4ca6-8755-be2a32b3c38c nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.746824913 +0000 UTC m=+139.313856022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls") pod "image-registry-546f954549-2sb84" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c") : secret "image-registry-tls" not found
Jul 17 14:51:52.746986 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.746844    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:54.746838552 +0000 UTC m=+125.313869662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:52.803656 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.803631    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bvflg_69b566a4-7ec7-483b-b348-2a7b08dcceef/dns-node-resolver/0.log"
Jul 17 14:51:52.847848 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.847815    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:51:52.848026 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:52.847864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:51:52.848026 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.847985    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 14:51:52.848026 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.847996    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 14:51:52.848155 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.848057    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert podName:bc284a53-0080-4ac6-9e8a-a46e170d7a87 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.848045218 +0000 UTC m=+139.415076329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-lkgf2" (UID: "bc284a53-0080-4ac6-9e8a-a46e170d7a87") : secret "networking-console-plugin-cert" not found
Jul 17 14:51:52.848155 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:52.848092    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls podName:0fcace03-2a71-44e3-9bea-c33d3b7c4c04 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:08.848085051 +0000 UTC m=+139.415116162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls") pod "dns-default-txmxh" (UID: "0fcace03-2a71-44e3-9bea-c33d3b7c4c04") : secret "dns-default-metrics-tls" not found
Jul 17 14:51:53.635972 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.635938    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-89rzb"]
Jul 17 14:51:53.700359 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.700319    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-89rzb"]
Jul 17 14:51:53.700516 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.700426    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.704015 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.703990    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 14:51:53.704177 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.704156    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ng6cn\""
Jul 17 14:51:53.705393 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.705376    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 14:51:53.705493 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.705478    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 14:51:53.705534 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.705491    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 14:51:53.756445 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.756405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-key\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.756819 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.756557    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-cabundle\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.756819 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.756583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94smw\" (UniqueName: \"kubernetes.io/projected/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-kube-api-access-94smw\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.857286 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.857259    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-cabundle\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.857286 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.857287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94smw\" (UniqueName: \"kubernetes.io/projected/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-kube-api-access-94smw\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.857469 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.857377    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-key\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.857863 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.857841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-cabundle\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.859649 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.859629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-signing-key\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:53.867002 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:53.866976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94smw\" (UniqueName: \"kubernetes.io/projected/eed12da1-3a39-4c44-a90e-11e8b0c1e19f-kube-api-access-94smw\") pod \"service-ca-58598695f7-89rzb\" (UID: \"eed12da1-3a39-4c44-a90e-11e8b0c1e19f\") " pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:54.003930 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.003848    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qmhfb_93e485cf-83b7-43d4-99bd-d9882c233d5b/node-ca/0.log"
Jul 17 14:51:54.009951 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.009932    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-89rzb"
Jul 17 14:51:54.145093 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.145049    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-89rzb"]
Jul 17 14:51:54.147952 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:54.147919    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed12da1_3a39_4c44_a90e_11e8b0c1e19f.slice/crio-6f1ee22cd784b75cc1935222e03588d24a8e509f8d00600e27640d8855f04d2e WatchSource:0}: Error finding container 6f1ee22cd784b75cc1935222e03588d24a8e509f8d00600e27640d8855f04d2e: Status 404 returned error can't find the container with id 6f1ee22cd784b75cc1935222e03588d24a8e509f8d00600e27640d8855f04d2e
Jul 17 14:51:54.413752 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.413716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-89rzb" event={"ID":"eed12da1-3a39-4c44-a90e-11e8b0c1e19f","Type":"ContainerStarted","Data":"9694bfe944f7c7aca3608c06d46bdeaa598b2f4510abcee3a41093227986d2bb"}
Jul 17 14:51:54.413940 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.413778    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-89rzb" event={"ID":"eed12da1-3a39-4c44-a90e-11e8b0c1e19f","Type":"ContainerStarted","Data":"6f1ee22cd784b75cc1935222e03588d24a8e509f8d00600e27640d8855f04d2e"}
Jul 17 14:51:54.430700 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.430648    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-89rzb" podStartSLOduration=1.430634328 podStartE2EDuration="1.430634328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:51:54.429758672 +0000 UTC m=+124.996789801" watchObservedRunningTime="2026-07-17 14:51:54.430634328 +0000 UTC m=+124.997665477"
Jul 17 14:51:54.765259 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.765170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:54.765743 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:54.765297    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 14:51:54.765743 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.765315    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:54.765743 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:54.765362    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:58.765346419 +0000 UTC m=+129.332377528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : configmap "kube-rbac-proxy" not found
Jul 17 14:51:54.765743 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:54.765442    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:54.765743 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:54.765499    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:51:58.765483322 +0000 UTC m=+129.332514439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:54.803596 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:54.803573    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rx4wv_a306a5b7-21a5-4135-9294-e48b5fb6793a/serve-healthcheck-canary/0.log"
Jul 17 14:51:57.134049 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:57.134013    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:57.134436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:57.134110    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:51:57.134436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:57.134419    2564 scope.go:117] "RemoveContainer" containerID="1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4"
Jul 17 14:51:57.134580 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:57.134559    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-jlhcq_openshift-console-operator(ddc5821f-f2b8-4072-9b11-abf6a3ac74ac)\"" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" podUID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac"
Jul 17 14:51:58.803158 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:58.803124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:58.803662 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:58.803282    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 14:51:58.803662 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:58.803306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:51:58.803662 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:58.803361    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:06.803339639 +0000 UTC m=+137.370370754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : configmap "kube-rbac-proxy" not found
Jul 17 14:51:58.803662 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:58.803419    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:58.803662 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:51:58.803466    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls podName:9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34 nodeName:}" failed. No retries permitted until 2026-07-17 14:52:06.803452523 +0000 UTC m=+137.370483632 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcnmv" (UID: "9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34") : secret "insights-runtime-extractor-tls" not found
Jul 17 14:51:59.610338 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:59.610300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:59.612780 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:59.612763    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 14:51:59.622975 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:59.622953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3f0b694-edc9-4e75-81de-7b528fa2195f-original-pull-secret\") pod \"global-pull-secret-syncer-6hdkd\" (UID: \"a3f0b694-edc9-4e75-81de-7b528fa2195f\") " pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:59.786983 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:59.786945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6hdkd"
Jul 17 14:51:59.948221 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:51:59.948195    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f0b694_edc9_4e75_81de_7b528fa2195f.slice/crio-5cb6cb79b07a898204abd1e0d59f79298fcc3463f3ed9ad02e87c9ef683eaf92 WatchSource:0}: Error finding container 5cb6cb79b07a898204abd1e0d59f79298fcc3463f3ed9ad02e87c9ef683eaf92: Status 404 returned error can't find the container with id 5cb6cb79b07a898204abd1e0d59f79298fcc3463f3ed9ad02e87c9ef683eaf92
Jul 17 14:51:59.949459 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:51:59.949438    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6hdkd"]
Jul 17 14:52:00.442946 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:00.442906    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6hdkd" event={"ID":"a3f0b694-edc9-4e75-81de-7b528fa2195f","Type":"ContainerStarted","Data":"5cb6cb79b07a898204abd1e0d59f79298fcc3463f3ed9ad02e87c9ef683eaf92"}
Jul 17 14:52:04.248500 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:04.248470    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lk2h9"
Jul 17 14:52:04.459563 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:04.459526    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6hdkd" event={"ID":"a3f0b694-edc9-4e75-81de-7b528fa2195f","Type":"ContainerStarted","Data":"8cf41607b8640a9999b2c190b3b6bbab9eb8ee5e7b31af56b7312e23c15397fb"}
Jul 17 14:52:04.475173 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:04.475121    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6hdkd" podStartSLOduration=130.847048403 podStartE2EDuration="2m14.475102995s" totalImagesPullingTime="3.628054592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:04.47325347 +0000 UTC m=+135.040284622" watchObservedRunningTime="2026-07-17 14:52:04.475102995 +0000 UTC m=+135.042134128"
Jul 17 14:52:06.876369 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:06.876333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:52:06.876747 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:06.876467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:52:06.876957 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:06.876937    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:52:06.878693 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:06.878676    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcnmv\" (UID: \"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34\") " pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:52:06.960809 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:06.960775    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pcnmv"
Jul 17 14:52:07.085229 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:07.085205    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pcnmv"]
Jul 17 14:52:07.087590 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:07.087559    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed7b49d_c3a0_4eaf_bf3a_54405c2c5c34.slice/crio-4a0b71c3d5adb87a527a0a7fc3fa4e1d08d644b29cfc4c0b93eb1a2902d16411 WatchSource:0}: Error finding container 4a0b71c3d5adb87a527a0a7fc3fa4e1d08d644b29cfc4c0b93eb1a2902d16411: Status 404 returned error can't find the container with id 4a0b71c3d5adb87a527a0a7fc3fa4e1d08d644b29cfc4c0b93eb1a2902d16411
Jul 17 14:52:07.476145 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:07.476100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcnmv" event={"ID":"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34","Type":"ContainerStarted","Data":"4a0b71c3d5adb87a527a0a7fc3fa4e1d08d644b29cfc4c0b93eb1a2902d16411"}
Jul 17 14:52:08.794358 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.794317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:52:08.794817 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.794415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:08.794817 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.794454    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:08.794817 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.794481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:52:08.794817 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.794519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:08.795394 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.795366    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-service-ca-bundle\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:08.797318 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.797274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"image-registry-546f954549-2sb84\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") " pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:08.797417 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.797396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5eb9ce6-f7ca-454c-900b-76b4cb6cae20-metrics-certs\") pod \"router-default-7c4d4bdd7d-ln92r\" (UID: \"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20\") " pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:08.797485 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.797396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f8126ee2-942d-4fea-a824-2e251bcf23b0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l5xqp\" (UID: \"f8126ee2-942d-4fea-a824-2e251bcf23b0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:52:08.797864 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.797842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/49279156-70fd-4c62-b3e3-2f02b820eecb-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-sg2zs\" (UID: \"49279156-70fd-4c62-b3e3-2f02b820eecb\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:52:08.896094 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.896038    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:52:08.896263 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.896185    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:52:08.898927 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.898899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0fcace03-2a71-44e3-9bea-c33d3b7c4c04-metrics-tls\") pod \"dns-default-txmxh\" (UID: \"0fcace03-2a71-44e3-9bea-c33d3b7c4c04\") " pod="openshift-dns/dns-default-txmxh"
Jul 17 14:52:08.899111 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.899094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bc284a53-0080-4ac6-9e8a-a46e170d7a87-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-lkgf2\" (UID: \"bc284a53-0080-4ac6-9e8a-a46e170d7a87\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:52:08.917468 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.917448    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-6sq5p\""
Jul 17 14:52:08.925189 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.925169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:08.925307 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.925195    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wbdf2\""
Jul 17 14:52:08.932919 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.932890    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"
Jul 17 14:52:08.945023 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.945004    2564 scope.go:117] "RemoveContainer" containerID="1e0cbee035dd5193b334f757273f6009f1d1dabf3a0a200fd05b8ec97bf2cff4"
Jul 17 14:52:08.949552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.949527    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9rrxw\""
Jul 17 14:52:08.957017 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.956986    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"
Jul 17 14:52:08.966262 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.966219    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mdwrv\""
Jul 17 14:52:08.974012 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:08.973982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:09.030411 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.030376    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hxssw\""
Jul 17 14:52:09.038473 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.038146    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-txmxh"
Jul 17 14:52:09.046836 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.046367    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2nqmv\""
Jul 17 14:52:09.061370 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.061141    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"
Jul 17 14:52:09.134997 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.134860    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c4d4bdd7d-ln92r"]
Jul 17 14:52:09.145195 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:09.145138    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5eb9ce6_f7ca_454c_900b_76b4cb6cae20.slice/crio-5ef5ec989b696d36e3c1481cdaee0ceb6445e772d93c97bf5c04ae0e82896f0c WatchSource:0}: Error finding container 5ef5ec989b696d36e3c1481cdaee0ceb6445e772d93c97bf5c04ae0e82896f0c: Status 404 returned error can't find the container with id 5ef5ec989b696d36e3c1481cdaee0ceb6445e772d93c97bf5c04ae0e82896f0c
Jul 17 14:52:09.187932 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.187836    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs"]
Jul 17 14:52:09.286241 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.286021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2"]
Jul 17 14:52:09.289116 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:09.289030    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc284a53_0080_4ac6_9e8a_a46e170d7a87.slice/crio-53fdae2ffe5004fa36575ec37713146d4a1163ecafe27b770c8c1b0d7860d0b1 WatchSource:0}: Error finding container 53fdae2ffe5004fa36575ec37713146d4a1163ecafe27b770c8c1b0d7860d0b1: Status 404 returned error can't find the container with id 53fdae2ffe5004fa36575ec37713146d4a1163ecafe27b770c8c1b0d7860d0b1
Jul 17 14:52:09.309384 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.309271    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-txmxh"]
Jul 17 14:52:09.312539 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:09.312515    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcace03_2a71_44e3_9bea_c33d3b7c4c04.slice/crio-5e81270c95dfc42cd1ea6b05958f188f3fda6c99e9652e48f5da8f408d44d05c WatchSource:0}: Error finding container 5e81270c95dfc42cd1ea6b05958f188f3fda6c99e9652e48f5da8f408d44d05c: Status 404 returned error can't find the container with id 5e81270c95dfc42cd1ea6b05958f188f3fda6c99e9652e48f5da8f408d44d05c
Jul 17 14:52:09.410939 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.410867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:52:09.413474 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.413415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp"]
Jul 17 14:52:09.415089 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:09.415043    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ad06c7_71b9_4ca6_8755_be2a32b3c38c.slice/crio-115c901b02f9015f77de54b516eb8c47c3070c2b44d80aea50097d23202f2b8e WatchSource:0}: Error finding container 115c901b02f9015f77de54b516eb8c47c3070c2b44d80aea50097d23202f2b8e: Status 404 returned error can't find the container with id 115c901b02f9015f77de54b516eb8c47c3070c2b44d80aea50097d23202f2b8e
Jul 17 14:52:09.416273 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:09.416246    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8126ee2_942d_4fea_a824_2e251bcf23b0.slice/crio-fa6bbbf92601b855dc9f85b457e38fb8d40f8422c9076beb4c397d4ca3b57969 WatchSource:0}: Error finding container fa6bbbf92601b855dc9f85b457e38fb8d40f8422c9076beb4c397d4ca3b57969: Status 404 returned error can't find the container with id fa6bbbf92601b855dc9f85b457e38fb8d40f8422c9076beb4c397d4ca3b57969
Jul 17 14:52:09.486180 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.486141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2" event={"ID":"bc284a53-0080-4ac6-9e8a-a46e170d7a87","Type":"ContainerStarted","Data":"53fdae2ffe5004fa36575ec37713146d4a1163ecafe27b770c8c1b0d7860d0b1"}
Jul 17 14:52:09.488301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.488282    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:52:09.488443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.488356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" event={"ID":"ddc5821f-f2b8-4072-9b11-abf6a3ac74ac","Type":"ContainerStarted","Data":"fa080d2eb1797610dd0a689b95276b58983dea48ffb5dba772f2da8c207b537a"}
Jul 17 14:52:09.488764 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.488747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:52:09.489720 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.489689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txmxh" event={"ID":"0fcace03-2a71-44e3-9bea-c33d3b7c4c04","Type":"ContainerStarted","Data":"5e81270c95dfc42cd1ea6b05958f188f3fda6c99e9652e48f5da8f408d44d05c"}
Jul 17 14:52:09.490947 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.490925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-546f954549-2sb84" event={"ID":"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c","Type":"ContainerStarted","Data":"115c901b02f9015f77de54b516eb8c47c3070c2b44d80aea50097d23202f2b8e"}
Jul 17 14:52:09.492309 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.492241    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp" event={"ID":"f8126ee2-942d-4fea-a824-2e251bcf23b0","Type":"ContainerStarted","Data":"fa6bbbf92601b855dc9f85b457e38fb8d40f8422c9076beb4c397d4ca3b57969"}
Jul 17 14:52:09.494714 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.494693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r" event={"ID":"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20","Type":"ContainerStarted","Data":"400c4b6f91c64e60a3a7c660b3dbe87cd807ef0dc13031f708a62390bb2354e8"}
Jul 17 14:52:09.494811 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.494732    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r" event={"ID":"f5eb9ce6-f7ca-454c-900b-76b4cb6cae20","Type":"ContainerStarted","Data":"5ef5ec989b696d36e3c1481cdaee0ceb6445e772d93c97bf5c04ae0e82896f0c"}
Jul 17 14:52:09.495886 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.495866    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs" event={"ID":"49279156-70fd-4c62-b3e3-2f02b820eecb","Type":"ContainerStarted","Data":"737f0e533417e0733b0b5e9364a3af31fafe91aa03502d8946207406c17ad4a2"}
Jul 17 14:52:09.505665 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.505620    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" podStartSLOduration=139.972435681 podStartE2EDuration="2m31.505602905s" totalImagesPullingTime="11.533167224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:09.504685013 +0000 UTC m=+140.071716191" watchObservedRunningTime="2026-07-17 14:52:09.505602905 +0000 UTC m=+140.072634038"
Jul 17 14:52:09.522436 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.522398    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r" podStartSLOduration=175.522385759 podStartE2EDuration="2m55.522385759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:09.520556162 +0000 UTC m=+140.087587306" watchObservedRunningTime="2026-07-17 14:52:09.522385759 +0000 UTC m=+140.089416891"
Jul 17 14:52:09.926080 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.926044    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:09.929012 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:09.928989    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:10.489261 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:10.489213    2564 patch_prober.go:28] interesting pod/console-operator-565b7b569d-jlhcq container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.9:8443/readyz\": context deadline exceeded" start-of-body=
Jul 17 14:52:10.489534 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:10.489310    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq" podUID="ddc5821f-f2b8-4072-9b11-abf6a3ac74ac" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.9:8443/readyz\": context deadline exceeded"
Jul 17 14:52:10.499721 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:10.499686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:10.501046 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:10.501022    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c4d4bdd7d-ln92r"
Jul 17 14:52:10.831700 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:10.831676    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-jlhcq"
Jul 17 14:52:12.510420 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:12.509940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-546f954549-2sb84" event={"ID":"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c","Type":"ContainerStarted","Data":"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"}
Jul 17 14:52:12.535282 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:12.535188    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-546f954549-2sb84" podStartSLOduration=148.535164363 podStartE2EDuration="2m28.535164363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:12.532661202 +0000 UTC m=+143.099692374" watchObservedRunningTime="2026-07-17 14:52:12.535164363 +0000 UTC m=+143.102195497"
Jul 17 14:52:13.515112 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:13.515080    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:14.522724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.522634    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcnmv" event={"ID":"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34","Type":"ContainerStarted","Data":"3e58091c04a8c7ab991fef883b11e59c784dc0b66caeed6e56d21234ccaa661b"}
Jul 17 14:52:14.525873 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.525046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs" event={"ID":"49279156-70fd-4c62-b3e3-2f02b820eecb","Type":"ContainerStarted","Data":"36dbe775784d351f511fa4ff99c5a2d00c791a7dd1a64b3309e841b9e7481e71"}
Jul 17 14:52:14.525873 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.525114    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs" event={"ID":"49279156-70fd-4c62-b3e3-2f02b820eecb","Type":"ContainerStarted","Data":"04d3989b9cef0f2261f2268bbc3829857925e0137f405879656344aef5d0cf01"}
Jul 17 14:52:14.528226 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.528155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txmxh" event={"ID":"0fcace03-2a71-44e3-9bea-c33d3b7c4c04","Type":"ContainerStarted","Data":"8cde0096847d933dafcf18199271a5723450cc190a1027ce5cce0005af7fc596"}
Jul 17 14:52:14.528226 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.528202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-txmxh" event={"ID":"0fcace03-2a71-44e3-9bea-c33d3b7c4c04","Type":"ContainerStarted","Data":"19b1174cf8468afc075d766378f5a52258498f389bb2fbde7189b08d84a0025d"}
Jul 17 14:52:14.528715 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.528696    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-txmxh"
Jul 17 14:52:14.531356 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.531336    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp" event={"ID":"f8126ee2-942d-4fea-a824-2e251bcf23b0","Type":"ContainerStarted","Data":"060de79c498921292bc506116e1c8385d1a4c6603872be85f964feb40ea4e1bd"}
Jul 17 14:52:14.548883 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.548605    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-sg2zs" podStartSLOduration=151.84468329 podStartE2EDuration="2m36.548586525s" totalImagesPullingTime="4.703903235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:14.548248075 +0000 UTC m=+145.115279207" watchObservedRunningTime="2026-07-17 14:52:14.548586525 +0000 UTC m=+145.115617657"
Jul 17 14:52:14.568901 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.568461    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l5xqp" podStartSLOduration=152.022276196 podStartE2EDuration="2m36.568445816s" totalImagesPullingTime="4.54616962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:14.568242154 +0000 UTC m=+145.135273287" watchObservedRunningTime="2026-07-17 14:52:14.568445816 +0000 UTC m=+145.135476948"
Jul 17 14:52:14.589166 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:14.588979    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-txmxh" podStartSLOduration=33.942883367 podStartE2EDuration="38.588960381s" totalImagesPullingTime="4.646077014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:14.586690388 +0000 UTC m=+145.153721522" watchObservedRunningTime="2026-07-17 14:52:14.588960381 +0000 UTC m=+145.155991515"
Jul 17 14:52:17.159155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:17.159120    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:52:17.545123 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:17.545090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcnmv" event={"ID":"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34","Type":"ContainerStarted","Data":"6cd4180e32c7cd2581f00e9d2c15968a9a11332a52809d55f606e3319c3fd2b6"}
Jul 17 14:52:19.656054 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:19.655966    2564 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b: reading manifest sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b in quay.io/openshift-release-dev/ocp-v4.0-art-dev: received unexpected HTTP status: 504 Gateway Time-out; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b"
Jul 17 14:52:19.656576 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:19.656131    2564 kuberuntime_manager.go:1459] "Unhandled Error" err="container networking-console-plugin start failed in pod networking-console-plugin-7bc7489cb5-lkgf2_openshift-network-console(bc284a53-0080-4ac6-9e8a-a46e170d7a87): ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b: reading manifest sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b in quay.io/openshift-release-dev/ocp-v4.0-art-dev: received unexpected HTTP status: 504 Gateway Time-out; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 17 14:52:19.656576 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:19.656177    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"networking-console-plugin\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b: reading manifest sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b in quay.io/openshift-release-dev/ocp-v4.0-art-dev: received unexpected HTTP status: 504 Gateway Time-out; artifact err: provided artifact is a container image\"" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2" podUID="bc284a53-0080-4ac6-9e8a-a46e170d7a87"
Jul 17 14:52:20.402157 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:20.402120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xqvs7"
Jul 17 14:52:20.560880 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:20.560840    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"networking-console-plugin\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: initializing source docker://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b: reading manifest sha256:48e25a47c9073c25fb7dcd8bb97a028316d1b002bc7b1efbc98ca1d6cbb7561b in quay.io/openshift-release-dev/ocp-v4.0-art-dev: received unexpected HTTP status: 504 Gateway Time-out; artifact err: provided artifact is a container image\"" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2" podUID="bc284a53-0080-4ac6-9e8a-a46e170d7a87"
Jul 17 14:52:21.564738 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:21.564699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcnmv" event={"ID":"9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34","Type":"ContainerStarted","Data":"fd1b6fc67e6af26b212c7cc4e1411deb35edd7889a8e2e65748f51fd17abb5d7"}
Jul 17 14:52:21.582816 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:21.582776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pcnmv" podStartSLOduration=17.000934546 podStartE2EDuration="30.582762217s" totalImagesPullingTime="13.581827671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 14:51:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:21.580743524 +0000 UTC m=+152.147774671" watchObservedRunningTime="2026-07-17 14:52:21.582762217 +0000 UTC m=+152.149793349"
Jul 17 14:52:25.542209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:25.542177    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-txmxh"
Jul 17 14:52:27.084564 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.084525    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x8tjt"]
Jul 17 14:52:27.088306 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.088287    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.090907 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.090876    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 14:52:27.091535 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.091516    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 14:52:27.091639 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.091544    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 14:52:27.091639 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.091610    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bbgng\""
Jul 17 14:52:27.091862 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.091848    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 14:52:27.153055 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfgp8\" (UniqueName: \"kubernetes.io/projected/06533c3f-ba49-462e-b824-9f5b0e28ac3b-kube-api-access-nfgp8\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153084    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153108    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-sys\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-wtmp\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-root\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153194    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-metrics-client-ca\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153223 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153529 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.153529 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.153333    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-textfile\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.166506 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.166486    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:27.254217 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-textfile\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254388 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfgp8\" (UniqueName: \"kubernetes.io/projected/06533c3f-ba49-462e-b824-9f5b0e28ac3b-kube-api-access-nfgp8\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254456 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254510 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254458    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-sys\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254510 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-wtmp\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-root\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254547    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-sys\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254564    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-metrics-client-ca\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254567    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-textfile\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254603 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-root\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.254647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-wtmp\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.254807 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:27.254744    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 14:52:27.254807 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:27.254805    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls podName:06533c3f-ba49-462e-b824-9f5b0e28ac3b nodeName:}" failed. No retries permitted until 2026-07-17 14:52:27.754787262 +0000 UTC m=+158.321818374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls") pod "node-exporter-x8tjt" (UID: "06533c3f-ba49-462e-b824-9f5b0e28ac3b") : secret "node-exporter-tls" not found
Jul 17 14:52:27.255100 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.255060    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.255197 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.255106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06533c3f-ba49-462e-b824-9f5b0e28ac3b-metrics-client-ca\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.257012 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.256989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.269231 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.269211    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfgp8\" (UniqueName: \"kubernetes.io/projected/06533c3f-ba49-462e-b824-9f5b0e28ac3b-kube-api-access-nfgp8\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.758074 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.758027    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.760333 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.760310    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06533c3f-ba49-462e-b824-9f5b0e28ac3b-node-exporter-tls\") pod \"node-exporter-x8tjt\" (UID: \"06533c3f-ba49-462e-b824-9f5b0e28ac3b\") " pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:27.998874 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:27.998842    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x8tjt"
Jul 17 14:52:28.011002 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:28.010935    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06533c3f_ba49_462e_b824_9f5b0e28ac3b.slice/crio-ab13cb1c07b80ea5ef994875c1dee45233e9b1471041869303c8b32e92700fac WatchSource:0}: Error finding container ab13cb1c07b80ea5ef994875c1dee45233e9b1471041869303c8b32e92700fac: Status 404 returned error can't find the container with id ab13cb1c07b80ea5ef994875c1dee45233e9b1471041869303c8b32e92700fac
Jul 17 14:52:28.592257 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.592221    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8tjt" event={"ID":"06533c3f-ba49-462e-b824-9f5b0e28ac3b","Type":"ContainerStarted","Data":"ab13cb1c07b80ea5ef994875c1dee45233e9b1471041869303c8b32e92700fac"}
Jul 17 14:52:28.867921 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.867831    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:52:28.870458 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.870434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 14:52:28.881210 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.881188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea32a2a8-657e-498b-a573-0c85ffa9eec3-metrics-certs\") pod \"network-metrics-daemon-c8d6d\" (UID: \"ea32a2a8-657e-498b-a573-0c85ffa9eec3\") " pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:52:28.894437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.894409    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7594c\""
Jul 17 14:52:28.902379 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:28.902357    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8d6d"
Jul 17 14:52:29.103284 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:29.103252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c8d6d"]
Jul 17 14:52:29.107295 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:52:29.107268    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea32a2a8_657e_498b_a573_0c85ffa9eec3.slice/crio-fd1fc7849c7cb6bd8bcd3188f365070d64568767b7080a414152a70f72d4be96 WatchSource:0}: Error finding container fd1fc7849c7cb6bd8bcd3188f365070d64568767b7080a414152a70f72d4be96: Status 404 returned error can't find the container with id fd1fc7849c7cb6bd8bcd3188f365070d64568767b7080a414152a70f72d4be96
Jul 17 14:52:29.597726 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:29.597688    2564 generic.go:356] "Generic (PLEG): container finished" podID="06533c3f-ba49-462e-b824-9f5b0e28ac3b" containerID="9ed199a46f1b25f94d4966be431b7030f50aacf538d4a7ed9838c46646597e98" exitCode=0
Jul 17 14:52:29.598184 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:29.597781    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8tjt" event={"ID":"06533c3f-ba49-462e-b824-9f5b0e28ac3b","Type":"ContainerDied","Data":"9ed199a46f1b25f94d4966be431b7030f50aacf538d4a7ed9838c46646597e98"}
Jul 17 14:52:29.599279 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:29.599240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8d6d" event={"ID":"ea32a2a8-657e-498b-a573-0c85ffa9eec3","Type":"ContainerStarted","Data":"fd1fc7849c7cb6bd8bcd3188f365070d64568767b7080a414152a70f72d4be96"}
Jul 17 14:52:30.605560 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.605529    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8tjt" event={"ID":"06533c3f-ba49-462e-b824-9f5b0e28ac3b","Type":"ContainerStarted","Data":"fdd7e1ca4f99fbda662a1cb9fee7705ff528dac224b3dbf3730dc9fd66ae8e18"}
Jul 17 14:52:30.605986 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.605586    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x8tjt" event={"ID":"06533c3f-ba49-462e-b824-9f5b0e28ac3b","Type":"ContainerStarted","Data":"16099c2e910061b915deb90f14afc6caa226781c23bdc416ba6858a554c7fea7"}
Jul 17 14:52:30.607189 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.607167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8d6d" event={"ID":"ea32a2a8-657e-498b-a573-0c85ffa9eec3","Type":"ContainerStarted","Data":"6667a7b7d12386a3d91ca5af17bff6ec333639a7dc891836cd880d4c346ac2a0"}
Jul 17 14:52:30.607250 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.607212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8d6d" event={"ID":"ea32a2a8-657e-498b-a573-0c85ffa9eec3","Type":"ContainerStarted","Data":"f933d6dde7c94fca9e85282509a00f4f9dbbb5458b1ec1b6ff3f14c1565c76d4"}
Jul 17 14:52:30.633772 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.633732    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x8tjt" podStartSLOduration=2.57857295 podStartE2EDuration="3.633720146s" totalImagesPullingTime="987.669614ms" totalInitContainerRuntime="67.477582ms" isStatefulPod=true podCreationTimestamp="2026-07-17 14:52:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:30.632779162 +0000 UTC m=+161.199810295" watchObservedRunningTime="2026-07-17 14:52:30.633720146 +0000 UTC m=+161.200751278"
Jul 17 14:52:30.657262 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:30.657223    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c8d6d" podStartSLOduration=129.699000972 podStartE2EDuration="2m10.657211832s" totalImagesPullingTime="958.21086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:50:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:52:30.656076083 +0000 UTC m=+161.223107210" watchObservedRunningTime="2026-07-17 14:52:30.657211832 +0000 UTC m=+161.224242995"
Jul 17 14:52:36.632019 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:36.631982    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2" event={"ID":"bc284a53-0080-4ac6-9e8a-a46e170d7a87","Type":"ContainerStarted","Data":"6102fbc28d7bf5a6cbe8a3ec8c056c0607fc170cff839c9fc116d7c0a8cfb4f7"}
Jul 17 14:52:36.655272 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:36.655228    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-lkgf2" podStartSLOduration=92.722505639 podStartE2EDuration="1m59.65521153s" totalImagesPullingTime="26.932705891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:50:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-17 14:52:36.6537545 +0000 UTC m=+167.220785633" watchObservedRunningTime="2026-07-17 14:52:36.65521153 +0000 UTC m=+167.222242661"
Jul 17 14:52:42.184725 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.184584    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-546f954549-2sb84" podUID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" containerName="registry" containerID="cri-o://33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c" gracePeriod=30
Jul 17 14:52:42.452906 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.452883    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:42.585909 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.585881    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.585909 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.585910    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.585980    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586034    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586093    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5wwq\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586157    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586454 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586187    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca\") pod \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\" (UID: \"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c\") "
Jul 17 14:52:42.586703 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586647    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:52:42.586801 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.586753    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:52:42.588536 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.588508    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:52:42.588649 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.588625    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:52:42.588793 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.588761    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:52:42.588959 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.588794    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:52:42.588959 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.588863    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq" (OuterVolumeSpecName: "kube-api-access-r5wwq") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "kube-api-access-r5wwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:52:42.594645 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.594618    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" (UID: "c5ad06c7-71b9-4ca6-8755-be2a32b3c38c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:52:42.655488 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.655461    2564 generic.go:356] "Generic (PLEG): container finished" podID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" containerID="33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c" exitCode=0
Jul 17 14:52:42.655625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.655501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-546f954549-2sb84" event={"ID":"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c","Type":"ContainerDied","Data":"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"}
Jul 17 14:52:42.655625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.655542    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-546f954549-2sb84" event={"ID":"c5ad06c7-71b9-4ca6-8755-be2a32b3c38c","Type":"ContainerDied","Data":"115c901b02f9015f77de54b516eb8c47c3070c2b44d80aea50097d23202f2b8e"}
Jul 17 14:52:42.655625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.655561    2564 scope.go:117] "RemoveContainer" containerID="33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"
Jul 17 14:52:42.655625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.655505    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-546f954549-2sb84"
Jul 17 14:52:42.666041 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.665796    2564 scope.go:117] "RemoveContainer" containerID="33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"
Jul 17 14:52:42.666683 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:52:42.666474    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c\": container with ID starting with 33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c not found: ID does not exist" containerID="33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"
Jul 17 14:52:42.666683 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.666530    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c"} err="failed to get container status \"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c\": rpc error: code = NotFound desc = could not find container \"33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c\": container with ID starting with 33ae4da30535d52a1e71d86e6a4bf22381477114cd682392f0548e65fcb1440c not found: ID does not exist"
Jul 17 14:52:42.683161 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.683135    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:52:42.687168 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687110    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5wwq\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-kube-api-access-r5wwq\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687168 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687141    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-ca-trust-extracted\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687168 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687154    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-trusted-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687168 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687166    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-installation-pull-secrets\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687179    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687192    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-bound-sa-token\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687204    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-image-registry-private-configuration\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.687423 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.687212    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c-registry-certificates\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:52:42.688520 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:42.688498    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-546f954549-2sb84"]
Jul 17 14:52:43.949137 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:52:43.949104    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" path="/var/lib/kubelet/pods/c5ad06c7-71b9-4ca6-8755-be2a32b3c38c/volumes"
Jul 17 14:53:00.723244 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:00.723207    2564 generic.go:356] "Generic (PLEG): container finished" podID="26575325-fed3-4634-8f77-278b4bd35d45" containerID="df0df311bf244f628682bae88eb712e333ed3d4d8e9ae119651c3bbcf1348db8" exitCode=0
Jul 17 14:53:00.723589 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:00.723279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ls4vc" event={"ID":"26575325-fed3-4634-8f77-278b4bd35d45","Type":"ContainerDied","Data":"df0df311bf244f628682bae88eb712e333ed3d4d8e9ae119651c3bbcf1348db8"}
Jul 17 14:53:00.723653 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:00.723639    2564 scope.go:117] "RemoveContainer" containerID="df0df311bf244f628682bae88eb712e333ed3d4d8e9ae119651c3bbcf1348db8"
Jul 17 14:53:01.370787 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:01.370758    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4d4bdd7d-ln92r_f5eb9ce6-f7ca-454c-900b-76b4cb6cae20/router/0.log"
Jul 17 14:53:01.389263 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:01.389232    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rx4wv_a306a5b7-21a5-4135-9294-e48b5fb6793a/serve-healthcheck-canary/0.log"
Jul 17 14:53:01.728607 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:01.728524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ls4vc" event={"ID":"26575325-fed3-4634-8f77-278b4bd35d45","Type":"ContainerStarted","Data":"48e2e894c94c2fa36e0d620737551f946739737f4b81a0af35ff22e469b2a29f"}
Jul 17 14:53:05.744332 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:05.744300    2564 generic.go:356] "Generic (PLEG): container finished" podID="a0baed11-a685-4de7-b51d-2316e2feba8b" containerID="ae2e57e3795e0c5cb23722f3d83ab8144fdd142a73e8e5ae923f2a03db9615da" exitCode=0
Jul 17 14:53:05.744770 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:05.744370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975" event={"ID":"a0baed11-a685-4de7-b51d-2316e2feba8b","Type":"ContainerDied","Data":"ae2e57e3795e0c5cb23722f3d83ab8144fdd142a73e8e5ae923f2a03db9615da"}
Jul 17 14:53:05.744770 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:05.744678    2564 scope.go:117] "RemoveContainer" containerID="ae2e57e3795e0c5cb23722f3d83ab8144fdd142a73e8e5ae923f2a03db9615da"
Jul 17 14:53:06.750121 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:06.750082    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nk975" event={"ID":"a0baed11-a685-4de7-b51d-2316e2feba8b","Type":"ContainerStarted","Data":"6d15a95f5de356ac312779784de566bc42dadd879e0077f1e59e4c3cf43dd475"}
Jul 17 14:53:15.789254 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:15.789216    2564 generic.go:356] "Generic (PLEG): container finished" podID="a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c" containerID="042734cb360959fb9f7a1e123772135211c469ac0b7d89b975d442556bf35c79" exitCode=0
Jul 17 14:53:15.789765 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:15.789259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7" event={"ID":"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c","Type":"ContainerDied","Data":"042734cb360959fb9f7a1e123772135211c469ac0b7d89b975d442556bf35c79"}
Jul 17 14:53:15.789765 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:15.789632    2564 scope.go:117] "RemoveContainer" containerID="042734cb360959fb9f7a1e123772135211c469ac0b7d89b975d442556bf35c79"
Jul 17 14:53:16.794818 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:16.794786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-7j5w7" event={"ID":"a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c","Type":"ContainerStarted","Data":"478bcd50e492f17a5a72ad5a5b0189741606ad8d3b121556c7c075df1fbccbf3"}
Jul 17 14:53:27.263532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:27.263488    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" podUID="76714e7f-b424-4427-8d5b-b1f681dd35b0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 17 14:53:37.263341 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:37.263301    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" podUID="76714e7f-b424-4427-8d5b-b1f681dd35b0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 17 14:53:47.263414 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.263379    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" podUID="76714e7f-b424-4427-8d5b-b1f681dd35b0" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 17 14:53:47.263838 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.263451    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn"
Jul 17 14:53:47.263899 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.263882    2564 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"d17ae4367c8745eb3af381ae38c18729d6aa0d62a5ad9593a06dd9bbc188eb82"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 17 14:53:47.263937 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.263919    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" podUID="76714e7f-b424-4427-8d5b-b1f681dd35b0" containerName="service-proxy" containerID="cri-o://d17ae4367c8745eb3af381ae38c18729d6aa0d62a5ad9593a06dd9bbc188eb82" gracePeriod=30
Jul 17 14:53:47.909320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.909286    2564 generic.go:356] "Generic (PLEG): container finished" podID="76714e7f-b424-4427-8d5b-b1f681dd35b0" containerID="d17ae4367c8745eb3af381ae38c18729d6aa0d62a5ad9593a06dd9bbc188eb82" exitCode=2
Jul 17 14:53:47.909494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.909359    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerDied","Data":"d17ae4367c8745eb3af381ae38c18729d6aa0d62a5ad9593a06dd9bbc188eb82"}
Jul 17 14:53:47.909494 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:53:47.909396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7bc9787d9b-b4bzn" event={"ID":"76714e7f-b424-4427-8d5b-b1f681dd35b0","Type":"ContainerStarted","Data":"1bb9a1e28c92e62f855c6539a1228814138e89a70912abbd61bfbea41a6e867d"}
Jul 17 14:54:49.820966 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:54:49.820938    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:54:49.826331 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:54:49.826308    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:54:49.829484 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:54:49.829460    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:54:49.830731 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:54:49.830715    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 14:54:49.835437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:54:49.835422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:57:10.001386 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.001353    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"]
Jul 17 14:57:10.001820 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.001634    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" containerName="registry"
Jul 17 14:57:10.001820 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.001645    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" containerName="registry"
Jul 17 14:57:10.001820 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.001706    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ad06c7-71b9-4ca6-8755-be2a32b3c38c" containerName="registry"
Jul 17 14:57:10.004421 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.004404    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.010840 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.010813    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 14:57:10.010954 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.010922    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 14:57:10.011044 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.010813    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 14:57:10.017343 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.017321    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 14:57:10.017435 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.017344    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ndbhs\""
Jul 17 14:57:10.033392 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.033369    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"]
Jul 17 14:57:10.135097 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.135046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.135252 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.135110    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.135252 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.135165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tq5n\" (UniqueName: \"kubernetes.io/projected/f94d6863-9b80-4d79-97e6-7813d72a4ac8-kube-api-access-2tq5n\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.135252 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.135229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-tmp\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.135366 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.135259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-manifests\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.236039 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.235994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tq5n\" (UniqueName: \"kubernetes.io/projected/f94d6863-9b80-4d79-97e6-7813d72a4ac8-kube-api-access-2tq5n\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.236209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.236051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-tmp\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.236252 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.236235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-manifests\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.236306 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.236290    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.236363 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.236345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.239225 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.237192    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-manifests\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.239225 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.238976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f94d6863-9b80-4d79-97e6-7813d72a4ac8-tmp\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.243113 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.239854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-webhook-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.243926 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.243902    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f94d6863-9b80-4d79-97e6-7813d72a4ac8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.251104 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.251082    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tq5n\" (UniqueName: \"kubernetes.io/projected/f94d6863-9b80-4d79-97e6-7813d72a4ac8-kube-api-access-2tq5n\") pod \"opendatahub-operator-controller-manager-dcf5668f8-bljrz\" (UID: \"f94d6863-9b80-4d79-97e6-7813d72a4ac8\") " pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.314590 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.314559    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:10.474667 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.474634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"]
Jul 17 14:57:10.479274 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:10.479246    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94d6863_9b80_4d79_97e6_7813d72a4ac8.slice/crio-fb86d36f7d75b5561290f4dab1cb3e640988aaac9cb4805d14d7ebe0f402776d WatchSource:0}: Error finding container fb86d36f7d75b5561290f4dab1cb3e640988aaac9cb4805d14d7ebe0f402776d: Status 404 returned error can't find the container with id fb86d36f7d75b5561290f4dab1cb3e640988aaac9cb4805d14d7ebe0f402776d
Jul 17 14:57:10.481040 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.481025    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 14:57:10.651341 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:10.651258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz" event={"ID":"f94d6863-9b80-4d79-97e6-7813d72a4ac8","Type":"ContainerStarted","Data":"fb86d36f7d75b5561290f4dab1cb3e640988aaac9cb4805d14d7ebe0f402776d"}
Jul 17 14:57:13.667547 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:13.667462    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz" event={"ID":"f94d6863-9b80-4d79-97e6-7813d72a4ac8","Type":"ContainerStarted","Data":"b5ea5fb683c0fc4af538bcda7f9d2be8ac40ccdc158c5eb7338216480a9625bd"}
Jul 17 14:57:14.672827 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:14.672736    2564 generic.go:356] "Generic (PLEG): container finished" podID="f94d6863-9b80-4d79-97e6-7813d72a4ac8" containerID="b5ea5fb683c0fc4af538bcda7f9d2be8ac40ccdc158c5eb7338216480a9625bd" exitCode=0
Jul 17 14:57:14.672827 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:14.672792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz" event={"ID":"f94d6863-9b80-4d79-97e6-7813d72a4ac8","Type":"ContainerDied","Data":"b5ea5fb683c0fc4af538bcda7f9d2be8ac40ccdc158c5eb7338216480a9625bd"}
Jul 17 14:57:15.679334 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:15.679302    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz" event={"ID":"f94d6863-9b80-4d79-97e6-7813d72a4ac8","Type":"ContainerStarted","Data":"9b63bfea142ab77d5047718cb781cffbe828977477a0934382039679682d4c56"}
Jul 17 14:57:15.679725 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:15.679393    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:15.708417 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:15.708371    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz" podStartSLOduration=2.409592646 podStartE2EDuration="6.708354091s" totalImagesPullingTime="3.260089707s" totalInitContainerRuntime="1.038671738s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:15.707776943 +0000 UTC m=+446.274808076" watchObservedRunningTime="2026-07-17 14:57:15.708354091 +0000 UTC m=+446.275385223"
Jul 17 14:57:26.685979 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:26.685948    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dcf5668f8-bljrz"
Jul 17 14:57:30.092045 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.092011    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"]
Jul 17 14:57:30.102052 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.102034    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.106199 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.106179    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zzdsf\""
Jul 17 14:57:30.107408 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.107373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 14:57:30.111730 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.111704    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 14:57:30.112362 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.111820    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"]
Jul 17 14:57:30.115602 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.115574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tmp\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.115760 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.115744    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtnz7\" (UniqueName: \"kubernetes.io/projected/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-kube-api-access-vtnz7\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.115889 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.115876    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tls-certs\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.217195 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.217162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tmp\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.217371 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.217217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtnz7\" (UniqueName: \"kubernetes.io/projected/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-kube-api-access-vtnz7\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.217371 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.217271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tls-certs\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.219682 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.219658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tmp\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.219920 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.219899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-tls-certs\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.234802 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.234771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtnz7\" (UniqueName: \"kubernetes.io/projected/8c1f7d77-954c-47d4-9f1a-78b0ee2912be-kube-api-access-vtnz7\") pod \"kube-auth-proxy-56cdb8d648-nn7d2\" (UID: \"8c1f7d77-954c-47d4-9f1a-78b0ee2912be\") " pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.415206 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.415125    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"
Jul 17 14:57:30.567144 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:30.567052    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1f7d77_954c_47d4_9f1a_78b0ee2912be.slice/crio-66b8c7e7a22a90fe8541b4c35a8f9e3daca671e5ad159ad3cb5a911ddce3d6a4 WatchSource:0}: Error finding container 66b8c7e7a22a90fe8541b4c35a8f9e3daca671e5ad159ad3cb5a911ddce3d6a4: Status 404 returned error can't find the container with id 66b8c7e7a22a90fe8541b4c35a8f9e3daca671e5ad159ad3cb5a911ddce3d6a4
Jul 17 14:57:30.585512 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.585480    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2"]
Jul 17 14:57:30.741746 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.741646    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2" event={"ID":"8c1f7d77-954c-47d4-9f1a-78b0ee2912be","Type":"ContainerStarted","Data":"66b8c7e7a22a90fe8541b4c35a8f9e3daca671e5ad159ad3cb5a911ddce3d6a4"}
Jul 17 14:57:30.774431 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.774397    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bz2bj"]
Jul 17 14:57:30.777518 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.777502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:30.781535 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.781517    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gm5kn\""
Jul 17 14:57:30.782823 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.782808    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 14:57:30.804747 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.804723    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bz2bj"]
Jul 17 14:57:30.821215 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.821187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:30.821345 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.821295    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljlb4\" (UniqueName: \"kubernetes.io/projected/fe2fce4c-57de-4432-affe-e6de27ac9166-kube-api-access-ljlb4\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:30.922317 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.922286    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:30.922520 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.922338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljlb4\" (UniqueName: \"kubernetes.io/projected/fe2fce4c-57de-4432-affe-e6de27ac9166-kube-api-access-ljlb4\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:30.922520 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:30.922462    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 17 14:57:30.922625 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:30.922540    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert podName:fe2fce4c-57de-4432-affe-e6de27ac9166 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:31.422517604 +0000 UTC m=+461.989548732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert") pod "odh-model-controller-7998c67dc4-bz2bj" (UID: "fe2fce4c-57de-4432-affe-e6de27ac9166") : secret "odh-model-controller-webhook-cert" not found
Jul 17 14:57:30.937889 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:30.937860    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljlb4\" (UniqueName: \"kubernetes.io/projected/fe2fce4c-57de-4432-affe-e6de27ac9166-kube-api-access-ljlb4\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:31.427506 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.427462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:31.430275 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.430245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe2fce4c-57de-4432-affe-e6de27ac9166-cert\") pod \"odh-model-controller-7998c67dc4-bz2bj\" (UID: \"fe2fce4c-57de-4432-affe-e6de27ac9166\") " pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:31.688139 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.688047    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:31.843321 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.843295    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bz2bj"]
Jul 17 14:57:31.846678 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:31.846641    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2fce4c_57de_4432_affe_e6de27ac9166.slice/crio-478f46617a86573c2d80d248b4aa9492820510464695a27e6f45784162d9cc59 WatchSource:0}: Error finding container 478f46617a86573c2d80d248b4aa9492820510464695a27e6f45784162d9cc59: Status 404 returned error can't find the container with id 478f46617a86573c2d80d248b4aa9492820510464695a27e6f45784162d9cc59
Jul 17 14:57:31.934773 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.934661    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xx49w"]
Jul 17 14:57:31.939667 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.939586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:31.944090 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.943860    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 14:57:31.944090 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.943953    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-trjfs\""
Jul 17 14:57:31.967009 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:31.966936    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xx49w"]
Jul 17 14:57:32.032473 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.032423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.032676 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.032516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mdr9\" (UniqueName: \"kubernetes.io/projected/95147308-7c75-4eca-a1d1-e0104f7ec585-kube-api-access-9mdr9\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.133638 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.133588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mdr9\" (UniqueName: \"kubernetes.io/projected/95147308-7c75-4eca-a1d1-e0104f7ec585-kube-api-access-9mdr9\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.133827 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.133675    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.133827 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.133801    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 14:57:32.133934 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.133861    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs podName:95147308-7c75-4eca-a1d1-e0104f7ec585 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:32.633840902 +0000 UTC m=+463.200872030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs") pod "model-serving-api-54d89bd5fc-xx49w" (UID: "95147308-7c75-4eca-a1d1-e0104f7ec585") : secret "model-serving-api-tls" not found
Jul 17 14:57:32.154696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.154647    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mdr9\" (UniqueName: \"kubernetes.io/projected/95147308-7c75-4eca-a1d1-e0104f7ec585-kube-api-access-9mdr9\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.166365 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.166333    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:32.169405 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.169384    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.182592 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.182564    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"istiod-openshift-gateway-7574459656-c2hc9\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.182592 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.182574    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cacerts\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret"
Jul 17 14:57:32.182792 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.182733    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-tls\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret"
Jul 17 14:57:32.182792 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.182749    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istio-kubeconfig\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret"
Jul 17 14:57:32.183739 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.183719    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-openshift-gateway-dockercfg-7ck9x\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7ck9x\"" type="*v1.Secret"
Jul 17 14:57:32.184075 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:32.184043    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap"
Jul 17 14:57:32.216251 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.216180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:32.234613 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234762 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234762 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234711    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234762 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234927 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgtv\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234927 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.234927 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.234904    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335703 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgtv\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.335917 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.335854    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.338610 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.338578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.348336 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.348267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.351720 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.351695    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgtv\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:32.640173 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.639585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.643109 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.643039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95147308-7c75-4eca-a1d1-e0104f7ec585-tls-certs\") pod \"model-serving-api-54d89bd5fc-xx49w\" (UID: \"95147308-7c75-4eca-a1d1-e0104f7ec585\") " pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:32.755207 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.755163    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj" event={"ID":"fe2fce4c-57de-4432-affe-e6de27ac9166","Type":"ContainerStarted","Data":"478f46617a86573c2d80d248b4aa9492820510464695a27e6f45784162d9cc59"}
Jul 17 14:57:32.855001 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:32.854967    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:33.139625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.139587    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 14:57:33.150710 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.150253    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336837    2564 secret.go:189] Couldn't get secret openshift-ingress/istiod-tls: failed to sync secret cache: timed out waiting for the condition
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336863    2564 secret.go:189] Couldn't get secret openshift-ingress/istio-kubeconfig: failed to sync secret cache: timed out waiting for the condition
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336835    2564 configmap.go:193] Couldn't get configMap openshift-ingress/openshift-gw-ca-root-cert: failed to sync configmap cache: timed out waiting for the condition
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336941    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig podName:5add070b-9245-4dfa-b410-c8e084755dd7 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:33.836917772 +0000 UTC m=+464.403948978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-kubeconfig" (UniqueName: "kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig") pod "istiod-openshift-gateway-7574459656-c2hc9" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7") : failed to sync secret cache: timed out waiting for the condition
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336962    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert podName:5add070b-9245-4dfa-b410-c8e084755dd7 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:33.836952332 +0000 UTC m=+464.403983447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-csr-dns-cert" (UniqueName: "kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert") pod "istiod-openshift-gateway-7574459656-c2hc9" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7") : failed to sync secret cache: timed out waiting for the condition
Jul 17 14:57:33.337210 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:33.336978    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap podName:5add070b-9245-4dfa-b410-c8e084755dd7 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:33.836969203 +0000 UTC m=+464.404000320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-csr-ca-configmap" (UniqueName: "kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap") pod "istiod-openshift-gateway-7574459656-c2hc9" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7") : failed to sync configmap cache: timed out waiting for the condition
Jul 17 14:57:33.434634 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.434563    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 14:57:33.469302 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.469271    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 14:57:33.502897 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.502869    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 14:57:33.593474 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.593446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7ck9x\""
Jul 17 14:57:33.853532 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.853494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.854017 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.853568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.854017 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.853618    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.854239 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.854208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.856521 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.856494    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.856634 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.856601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-c2hc9\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:33.979549 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:33.979519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:35.061962 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.061924    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-xx49w"]
Jul 17 14:57:35.065499 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:35.065461    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95147308_7c75_4eca_a1d1_e0104f7ec585.slice/crio-d06d08274d07b2e603c69207722917b08943c52b912014cb1c0b3a95e1e553a0 WatchSource:0}: Error finding container d06d08274d07b2e603c69207722917b08943c52b912014cb1c0b3a95e1e553a0: Status 404 returned error can't find the container with id d06d08274d07b2e603c69207722917b08943c52b912014cb1c0b3a95e1e553a0
Jul 17 14:57:35.071785 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.071763    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:35.082986 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:35.082957    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5add070b_9245_4dfa_b410_c8e084755dd7.slice/crio-e00367cae4773b5af62a303bd66d475293da7e010be2eb853ccc2f89805963e2 WatchSource:0}: Error finding container e00367cae4773b5af62a303bd66d475293da7e010be2eb853ccc2f89805963e2: Status 404 returned error can't find the container with id e00367cae4773b5af62a303bd66d475293da7e010be2eb853ccc2f89805963e2
Jul 17 14:57:35.775505 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.774940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj" event={"ID":"fe2fce4c-57de-4432-affe-e6de27ac9166","Type":"ContainerStarted","Data":"070303c38574c8f7ce55f3cac1279e64307bf5bae3c3468cda33caf9e12aaf03"}
Jul 17 14:57:35.775505 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.775352    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:35.777407 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.777351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2" event={"ID":"8c1f7d77-954c-47d4-9f1a-78b0ee2912be","Type":"ContainerStarted","Data":"685fd9e29e13cec215fee5ae12e6e847c589ce0c750ea449de76406eb2e23cb9"}
Jul 17 14:57:35.778789 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.778754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w" event={"ID":"95147308-7c75-4eca-a1d1-e0104f7ec585","Type":"ContainerStarted","Data":"d06d08274d07b2e603c69207722917b08943c52b912014cb1c0b3a95e1e553a0"}
Jul 17 14:57:35.780232 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.780206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" event={"ID":"5add070b-9245-4dfa-b410-c8e084755dd7","Type":"ContainerStarted","Data":"e00367cae4773b5af62a303bd66d475293da7e010be2eb853ccc2f89805963e2"}
Jul 17 14:57:35.846806 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.846744    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj" podStartSLOduration=2.772088392 podStartE2EDuration="5.846722942s" totalImagesPullingTime="3.07463455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:35.844906983 +0000 UTC m=+466.411938116" watchObservedRunningTime="2026-07-17 14:57:35.846722942 +0000 UTC m=+466.413754075"
Jul 17 14:57:35.880967 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:35.880909    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56cdb8d648-nn7d2" podStartSLOduration=1.5470632709999999 podStartE2EDuration="5.880891086s" totalImagesPullingTime="4.333827815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:35.879840275 +0000 UTC m=+466.446871409" watchObservedRunningTime="2026-07-17 14:57:35.880891086 +0000 UTC m=+466.447922253"
Jul 17 14:57:37.790523 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:37.790485    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w" event={"ID":"95147308-7c75-4eca-a1d1-e0104f7ec585","Type":"ContainerStarted","Data":"0915030bcfcf8d6b93123aa1042225eac0b2c4c2d3781826b24f3fb069691cad"}
Jul 17 14:57:37.790998 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:37.790672    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:37.812908 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:37.812860    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w" podStartSLOduration=5.112713215 podStartE2EDuration="6.812844043s" totalImagesPullingTime="1.700130828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:37.810766744 +0000 UTC m=+468.377797903" watchObservedRunningTime="2026-07-17 14:57:37.812844043 +0000 UTC m=+468.379875175"
Jul 17 14:57:41.376716 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:41.376669    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 14:57:41.376981 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:41.376762    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 14:57:41.811848 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:41.811813    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" event={"ID":"5add070b-9245-4dfa-b410-c8e084755dd7","Type":"ContainerStarted","Data":"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"}
Jul 17 14:57:41.812044 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:41.811967    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:41.854424 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:41.854364    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" podStartSLOduration=3.563158949 podStartE2EDuration="9.854344972s" totalImagesPullingTime="6.291186023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:41.854143103 +0000 UTC m=+472.421174235" watchObservedRunningTime="2026-07-17 14:57:41.854344972 +0000 UTC m=+472.421376105"
Jul 17 14:57:42.818724 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:42.818693    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-c2hc9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 14:57:42.819037 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:42.818756    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 14:57:43.294942 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.294851    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"]
Jul 17 14:57:43.298134 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.298112    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.302641 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.302612    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 14:57:43.302751 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.302653    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hw28w\""
Jul 17 14:57:43.315868 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.315841    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"]
Jul 17 14:57:43.329681 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.329655    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"]
Jul 17 14:57:43.332858 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.332837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.340595 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.340509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.340707 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.340634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwzbv\" (UniqueName: \"kubernetes.io/projected/77fa29ed-3d9f-45aa-bf56-d2899b788186-kube-api-access-qwzbv\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.344451 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.344095    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 14:57:43.344451 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.344358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nj2bj\""
Jul 17 14:57:43.371549 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.371513    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"]
Jul 17 14:57:43.441320 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.441288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.441498 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.441332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwzbv\" (UniqueName: \"kubernetes.io/projected/77fa29ed-3d9f-45aa-bf56-d2899b788186-kube-api-access-qwzbv\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.441498 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.441458    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hlmm\" (UniqueName: \"kubernetes.io/projected/584db22e-ee21-4b1b-9015-847365f25090-kube-api-access-7hlmm\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.441602 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.441508    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.441602 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:43.441584    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 17 14:57:43.441696 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:43.441659    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert podName:77fa29ed-3d9f-45aa-bf56-d2899b788186 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:43.941639485 +0000 UTC m=+474.508670609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert") pod "kserve-controller-manager-d6f4b65b8-4px7w" (UID: "77fa29ed-3d9f-45aa-bf56-d2899b788186") : secret "kserve-webhook-server-cert" not found
Jul 17 14:57:43.527731 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.527698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwzbv\" (UniqueName: \"kubernetes.io/projected/77fa29ed-3d9f-45aa-bf56-d2899b788186-kube-api-access-qwzbv\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.542878 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.542841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.543007 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.542932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hlmm\" (UniqueName: \"kubernetes.io/projected/584db22e-ee21-4b1b-9015-847365f25090-kube-api-access-7hlmm\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.543007 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:43.542964    2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 17 14:57:43.543121 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:43.543015    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert podName:584db22e-ee21-4b1b-9015-847365f25090 nodeName:}" failed. No retries permitted until 2026-07-17 14:57:44.043000175 +0000 UTC m=+474.610031285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert") pod "llmisvc-controller-manager-777c6bd7d5-6dzdn" (UID: "584db22e-ee21-4b1b-9015-847365f25090") : secret "llmisvc-webhook-server-cert" not found
Jul 17 14:57:43.607437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.607410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hlmm\" (UniqueName: \"kubernetes.io/projected/584db22e-ee21-4b1b-9015-847365f25090-kube-api-access-7hlmm\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:43.946664 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.946573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:43.949309 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:43.949286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77fa29ed-3d9f-45aa-bf56-d2899b788186-cert\") pod \"kserve-controller-manager-d6f4b65b8-4px7w\" (UID: \"77fa29ed-3d9f-45aa-bf56-d2899b788186\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:44.047202 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.047173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:44.049463 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.049442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584db22e-ee21-4b1b-9015-847365f25090-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-6dzdn\" (UID: \"584db22e-ee21-4b1b-9015-847365f25090\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:44.209522 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.209440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:44.248731 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.248698    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:44.367463 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.367439    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"]
Jul 17 14:57:44.369000 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:44.368978    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fa29ed_3d9f_45aa_bf56_d2899b788186.slice/crio-eec703c87f310a90f28e8fe167ec37d49b9dff74008b188756e48dcbeaf51f6f WatchSource:0}: Error finding container eec703c87f310a90f28e8fe167ec37d49b9dff74008b188756e48dcbeaf51f6f: Status 404 returned error can't find the container with id eec703c87f310a90f28e8fe167ec37d49b9dff74008b188756e48dcbeaf51f6f
Jul 17 14:57:44.474271 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.474192    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"]
Jul 17 14:57:44.480270 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:57:44.480249    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod584db22e_ee21_4b1b_9015_847365f25090.slice/crio-ba4e20d414d008c023f8ee14ba699eb6c7d1dc3f515a8fa3b70f7abb23975673 WatchSource:0}: Error finding container ba4e20d414d008c023f8ee14ba699eb6c7d1dc3f515a8fa3b70f7abb23975673: Status 404 returned error can't find the container with id ba4e20d414d008c023f8ee14ba699eb6c7d1dc3f515a8fa3b70f7abb23975673
Jul 17 14:57:44.825772 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.825740    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn" event={"ID":"584db22e-ee21-4b1b-9015-847365f25090","Type":"ContainerStarted","Data":"ba4e20d414d008c023f8ee14ba699eb6c7d1dc3f515a8fa3b70f7abb23975673"}
Jul 17 14:57:44.826807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:44.826785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w" event={"ID":"77fa29ed-3d9f-45aa-bf56-d2899b788186","Type":"ContainerStarted","Data":"eec703c87f310a90f28e8fe167ec37d49b9dff74008b188756e48dcbeaf51f6f"}
Jul 17 14:57:45.818162 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:45.818132    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:46.787747 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:46.787709    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bz2bj"
Jul 17 14:57:47.148498 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.148468    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:47.148939 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.148654    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" containerName="discovery" containerID="cri-o://7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31" gracePeriod=30
Jul 17 14:57:47.809337 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.809314    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:47.859600 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.859554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn" event={"ID":"584db22e-ee21-4b1b-9015-847365f25090","Type":"ContainerStarted","Data":"a86702d619f2fdbade3972b48df48a67c9481722726c6f9659d82ed0d76edab3"}
Jul 17 14:57:47.859805 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.859656    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:57:47.860925 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.860902    2564 generic.go:356] "Generic (PLEG): container finished" podID="5add070b-9245-4dfa-b410-c8e084755dd7" containerID="7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31" exitCode=0
Jul 17 14:57:47.861041 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.860949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"
Jul 17 14:57:47.861041 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.860974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" event={"ID":"5add070b-9245-4dfa-b410-c8e084755dd7","Type":"ContainerDied","Data":"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"}
Jul 17 14:57:47.861041 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.861019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9" event={"ID":"5add070b-9245-4dfa-b410-c8e084755dd7","Type":"ContainerDied","Data":"e00367cae4773b5af62a303bd66d475293da7e010be2eb853ccc2f89805963e2"}
Jul 17 14:57:47.861220 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.861090    2564 scope.go:117] "RemoveContainer" containerID="7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"
Jul 17 14:57:47.862657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.862632    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w" event={"ID":"77fa29ed-3d9f-45aa-bf56-d2899b788186","Type":"ContainerStarted","Data":"9da73ce867118150d6b56a3bd1af30fa36aac76ce2b38b32af8a76952b8702d9"}
Jul 17 14:57:47.862809 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.862795    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:57:47.869395 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.869380    2564 scope.go:117] "RemoveContainer" containerID="7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"
Jul 17 14:57:47.869661 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:57:47.869644    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31\": container with ID starting with 7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31 not found: ID does not exist" containerID="7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"
Jul 17 14:57:47.869712 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.869669    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31"} err="failed to get container status \"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31\": rpc error: code = NotFound desc = could not find container \"7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31\": container with ID starting with 7118630f35278b5a2275450d1ef308eaefdcc90d588442cd8c346e7f8d01ad31 not found: ID does not exist"
Jul 17 14:57:47.885318 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885292    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwgtv\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885347    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885377    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885396    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885414    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885436    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885694 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885498    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs\") pod \"5add070b-9245-4dfa-b410-c8e084755dd7\" (UID: \"5add070b-9245-4dfa-b410-c8e084755dd7\") "
Jul 17 14:57:47.885981 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.885920    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 14:57:47.886132 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.886048    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-ca-configmap\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.887778 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.887746    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:57:47.887869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.887813    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs" (OuterVolumeSpecName: "local-certs") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:57:47.888031 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.888007    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv" (OuterVolumeSpecName: "kube-api-access-xwgtv") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "kube-api-access-xwgtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:57:47.888148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.888132    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts" (OuterVolumeSpecName: "cacerts") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:57:47.888198 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.888177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:57:47.888198 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.888177    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5add070b-9245-4dfa-b410-c8e084755dd7" (UID: "5add070b-9245-4dfa-b410-c8e084755dd7"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 14:57:47.897861 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.897819    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn" podStartSLOduration=1.683971896 podStartE2EDuration="4.897804957s" totalImagesPullingTime="3.213833061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:47.89755499 +0000 UTC m=+478.464586124" watchObservedRunningTime="2026-07-17 14:57:47.897804957 +0000 UTC m=+478.464836080"
Jul 17 14:57:47.946232 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.946125    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w" podStartSLOduration=2.290354748 podStartE2EDuration="4.946105093s" totalImagesPullingTime="2.655750345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:57:47.945910543 +0000 UTC m=+478.512941676" watchObservedRunningTime="2026-07-17 14:57:47.946105093 +0000 UTC m=+478.513136227"
Jul 17 14:57:47.987107 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987055    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-kubeconfig\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.987107 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987104    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-cacerts\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.987107 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987113    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5add070b-9245-4dfa-b410-c8e084755dd7-istio-csr-dns-cert\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.987327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987122    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-istio-token\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.987327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987131    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5add070b-9245-4dfa-b410-c8e084755dd7-local-certs\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:47.987327 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:47.987139    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwgtv\" (UniqueName: \"kubernetes.io/projected/5add070b-9245-4dfa-b410-c8e084755dd7-kube-api-access-xwgtv\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:57:48.189621 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:48.189586    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:48.201886 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:48.201818    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-c2hc9"]
Jul 17 14:57:48.798263 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:48.798237    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-xx49w"
Jul 17 14:57:49.949865 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:57:49.949833    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" path="/var/lib/kubelet/pods/5add070b-9245-4dfa-b410-c8e084755dd7/volumes"
Jul 17 14:58:18.870874 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:18.870840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-6dzdn"
Jul 17 14:58:18.873944 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:18.873921    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4px7w"
Jul 17 14:58:31.212251 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.212222    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:31.212624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.212532    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" containerName="discovery"
Jul 17 14:58:31.212624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.212542    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" containerName="discovery"
Jul 17 14:58:31.212624 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.212600    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5add070b-9245-4dfa-b410-c8e084755dd7" containerName="discovery"
Jul 17 14:58:31.220902 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.220878    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:31.223960 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.223932    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 14:58:31.224181 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.223942    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kplzp\""
Jul 17 14:58:31.228077 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.228030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzz74\" (UniqueName: \"kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74\") pod \"kuadrant-operator-catalog-wqz7d\" (UID: \"08cf377a-2c62-4821-8b23-1ce25039526d\") " pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:31.231380 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.231354    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:31.231744 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.231723    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 14:58:31.328973 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.328941    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzz74\" (UniqueName: \"kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74\") pod \"kuadrant-operator-catalog-wqz7d\" (UID: \"08cf377a-2c62-4821-8b23-1ce25039526d\") " pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:31.339654 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.339627    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzz74\" (UniqueName: \"kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74\") pod \"kuadrant-operator-catalog-wqz7d\" (UID: \"08cf377a-2c62-4821-8b23-1ce25039526d\") " pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:31.534267 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.534175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:31.542631 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.542604    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:31.664253 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.664233    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:31.666434 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:58:31.666404    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08cf377a_2c62_4821_8b23_1ce25039526d.slice/crio-4de06b9300e781463cee3019264ac9d5fc2a762df70763c9176c0c137b1c87db WatchSource:0}: Error finding container 4de06b9300e781463cee3019264ac9d5fc2a762df70763c9176c0c137b1c87db: Status 404 returned error can't find the container with id 4de06b9300e781463cee3019264ac9d5fc2a762df70763c9176c0c137b1c87db
Jul 17 14:58:31.755296 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.755259    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qpbdr"]
Jul 17 14:58:31.758596 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.758575    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:31.769174 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.769153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qpbdr"]
Jul 17 14:58:31.831907 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.831874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcr44\" (UniqueName: \"kubernetes.io/projected/e37c9269-9fc3-4355-b79c-b69f47e59416-kube-api-access-jcr44\") pod \"kuadrant-operator-catalog-qpbdr\" (UID: \"e37c9269-9fc3-4355-b79c-b69f47e59416\") " pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:31.933203 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.933166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcr44\" (UniqueName: \"kubernetes.io/projected/e37c9269-9fc3-4355-b79c-b69f47e59416-kube-api-access-jcr44\") pod \"kuadrant-operator-catalog-qpbdr\" (UID: \"e37c9269-9fc3-4355-b79c-b69f47e59416\") " pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:31.946118 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:31.946059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcr44\" (UniqueName: \"kubernetes.io/projected/e37c9269-9fc3-4355-b79c-b69f47e59416-kube-api-access-jcr44\") pod \"kuadrant-operator-catalog-qpbdr\" (UID: \"e37c9269-9fc3-4355-b79c-b69f47e59416\") " pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:32.055654 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:32.055624    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" event={"ID":"08cf377a-2c62-4821-8b23-1ce25039526d","Type":"ContainerStarted","Data":"4de06b9300e781463cee3019264ac9d5fc2a762df70763c9176c0c137b1c87db"}
Jul 17 14:58:32.067870 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:32.067836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:32.202779 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:32.202747    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qpbdr"]
Jul 17 14:58:32.205623 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:58:32.205591    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37c9269_9fc3_4355_b79c_b69f47e59416.slice/crio-5747a156ddaef6fe3ebec72189118b447e5aef823c8638994ec5802be8d6152f WatchSource:0}: Error finding container 5747a156ddaef6fe3ebec72189118b447e5aef823c8638994ec5802be8d6152f: Status 404 returned error can't find the container with id 5747a156ddaef6fe3ebec72189118b447e5aef823c8638994ec5802be8d6152f
Jul 17 14:58:33.063321 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:33.063272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr" event={"ID":"e37c9269-9fc3-4355-b79c-b69f47e59416","Type":"ContainerStarted","Data":"5747a156ddaef6fe3ebec72189118b447e5aef823c8638994ec5802be8d6152f"}
Jul 17 14:58:34.069318 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.069290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr" event={"ID":"e37c9269-9fc3-4355-b79c-b69f47e59416","Type":"ContainerStarted","Data":"8699ee3a5917fa6b5de2608c70b10a2a20b7d3619812def72d53e8f59dbb2ab2"}
Jul 17 14:58:34.070650 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.070623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" event={"ID":"08cf377a-2c62-4821-8b23-1ce25039526d","Type":"ContainerStarted","Data":"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"}
Jul 17 14:58:34.070752 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.070707    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" podUID="08cf377a-2c62-4821-8b23-1ce25039526d" containerName="registry-server" containerID="cri-o://6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288" gracePeriod=2
Jul 17 14:58:34.088667 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.088626    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr" podStartSLOduration=1.56867216 podStartE2EDuration="3.088610668s" totalImagesPullingTime="1.519938508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:58:34.088592941 +0000 UTC m=+524.655624097" watchObservedRunningTime="2026-07-17 14:58:34.088610668 +0000 UTC m=+524.655641801"
Jul 17 14:58:34.109353 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.109310    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" podStartSLOduration=1.053071268 podStartE2EDuration="3.109295548s" totalImagesPullingTime="2.05622428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:58:34.10791251 +0000 UTC m=+524.674943662" watchObservedRunningTime="2026-07-17 14:58:34.109295548 +0000 UTC m=+524.676326681"
Jul 17 14:58:34.310904 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.310883    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:34.352999 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.352893    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzz74\" (UniqueName: \"kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74\") pod \"08cf377a-2c62-4821-8b23-1ce25039526d\" (UID: \"08cf377a-2c62-4821-8b23-1ce25039526d\") "
Jul 17 14:58:34.355258 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.355222    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74" (OuterVolumeSpecName: "kube-api-access-lzz74") pod "08cf377a-2c62-4821-8b23-1ce25039526d" (UID: "08cf377a-2c62-4821-8b23-1ce25039526d"). InnerVolumeSpecName "kube-api-access-lzz74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:58:34.453782 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:34.453745    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzz74\" (UniqueName: \"kubernetes.io/projected/08cf377a-2c62-4821-8b23-1ce25039526d-kube-api-access-lzz74\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:58:35.076155 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.076123    2564 generic.go:356] "Generic (PLEG): container finished" podID="08cf377a-2c62-4821-8b23-1ce25039526d" containerID="6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288" exitCode=0
Jul 17 14:58:35.076552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.076165    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d"
Jul 17 14:58:35.076552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.076216    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" event={"ID":"08cf377a-2c62-4821-8b23-1ce25039526d","Type":"ContainerDied","Data":"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"}
Jul 17 14:58:35.076552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.076276    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqz7d" event={"ID":"08cf377a-2c62-4821-8b23-1ce25039526d","Type":"ContainerDied","Data":"4de06b9300e781463cee3019264ac9d5fc2a762df70763c9176c0c137b1c87db"}
Jul 17 14:58:35.076552 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.076304    2564 scope.go:117] "RemoveContainer" containerID="6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"
Jul 17 14:58:35.085732 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.085716    2564 scope.go:117] "RemoveContainer" containerID="6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"
Jul 17 14:58:35.085990 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:58:35.085974    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288\": container with ID starting with 6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288 not found: ID does not exist" containerID="6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"
Jul 17 14:58:35.086041 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.085997    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288"} err="failed to get container status \"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288\": rpc error: code = NotFound desc = could not find container \"6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288\": container with ID starting with 6325ef15ed3b85894c92c3d225e11213a6ca0df0f3c9ef688a383227365da288 not found: ID does not exist"
Jul 17 14:58:35.100273 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.100251    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:35.107348 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.107328    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqz7d"]
Jul 17 14:58:35.950040 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:35.950010    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08cf377a-2c62-4821-8b23-1ce25039526d" path="/var/lib/kubelet/pods/08cf377a-2c62-4821-8b23-1ce25039526d/volumes"
Jul 17 14:58:42.068207 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:42.068113    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:42.068207 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:42.068189    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:42.090671 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:42.090646    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:58:42.128957 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:58:42.128931    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qpbdr"
Jul 17 14:59:00.237253 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.237221    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"]
Jul 17 14:59:00.237655 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.237532    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08cf377a-2c62-4821-8b23-1ce25039526d" containerName="registry-server"
Jul 17 14:59:00.237655 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.237544    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08cf377a-2c62-4821-8b23-1ce25039526d" containerName="registry-server"
Jul 17 14:59:00.237655 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.237616    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08cf377a-2c62-4821-8b23-1ce25039526d" containerName="registry-server"
Jul 17 14:59:00.241173 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.241155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:00.244665 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.244639    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-srkpr\""
Jul 17 14:59:00.244770 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.244745    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 14:59:00.256696 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.256669    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"]
Jul 17 14:59:00.376506 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.376464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdkm5\" (UniqueName: \"kubernetes.io/projected/cf917cbb-eec7-4eb7-ab60-e3213d93e12e-kube-api-access-bdkm5\") pod \"dns-operator-controller-manager-864dd78c4c-4vpnh\" (UID: \"cf917cbb-eec7-4eb7-ab60-e3213d93e12e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:00.477314 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.477276    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdkm5\" (UniqueName: \"kubernetes.io/projected/cf917cbb-eec7-4eb7-ab60-e3213d93e12e-kube-api-access-bdkm5\") pod \"dns-operator-controller-manager-864dd78c4c-4vpnh\" (UID: \"cf917cbb-eec7-4eb7-ab60-e3213d93e12e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:00.499403 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.499341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdkm5\" (UniqueName: \"kubernetes.io/projected/cf917cbb-eec7-4eb7-ab60-e3213d93e12e-kube-api-access-bdkm5\") pod \"dns-operator-controller-manager-864dd78c4c-4vpnh\" (UID: \"cf917cbb-eec7-4eb7-ab60-e3213d93e12e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:00.551210 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.551178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:00.706484 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:00.706459    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"]
Jul 17 14:59:00.709389 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:00.709357    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf917cbb_eec7_4eb7_ab60_e3213d93e12e.slice/crio-08fbe6779999155c3a3bb7ac89637697a328267e85af4e78a9aa230e6da576f9 WatchSource:0}: Error finding container 08fbe6779999155c3a3bb7ac89637697a328267e85af4e78a9aa230e6da576f9: Status 404 returned error can't find the container with id 08fbe6779999155c3a3bb7ac89637697a328267e85af4e78a9aa230e6da576f9
Jul 17 14:59:01.188354 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:01.188321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh" event={"ID":"cf917cbb-eec7-4eb7-ab60-e3213d93e12e","Type":"ContainerStarted","Data":"08fbe6779999155c3a3bb7ac89637697a328267e85af4e78a9aa230e6da576f9"}
Jul 17 14:59:03.200891 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.200855    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh" event={"ID":"cf917cbb-eec7-4eb7-ab60-e3213d93e12e","Type":"ContainerStarted","Data":"5db8d41e45ba5b54e007fd648fd204b8d19857bc797980bbcac0da0e8dba1130"}
Jul 17 14:59:03.201307 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.200973    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:03.242590 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.242535    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh" podStartSLOduration=1.458643843 podStartE2EDuration="3.242518463s" totalImagesPullingTime="1.78387462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:03.240965054 +0000 UTC m=+553.807996185" watchObservedRunningTime="2026-07-17 14:59:03.242518463 +0000 UTC m=+553.809549594"
Jul 17 14:59:03.776316 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.776278    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"]
Jul 17 14:59:03.778887 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.778868    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:03.781465 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.781440    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6vqk6\""
Jul 17 14:59:03.796435 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.796405    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"]
Jul 17 14:59:03.907437 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:03.907406    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dsbf\" (UniqueName: \"kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf\") pod \"limitador-operator-controller-manager-859d7b78bd-5knww\" (UID: \"369e56ad-d701-4496-b7cb-f6d0cf4c156e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:04.008351 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:04.008312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dsbf\" (UniqueName: \"kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf\") pod \"limitador-operator-controller-manager-859d7b78bd-5knww\" (UID: \"369e56ad-d701-4496-b7cb-f6d0cf4c156e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:04.026366 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:04.026303    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dsbf\" (UniqueName: \"kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf\") pod \"limitador-operator-controller-manager-859d7b78bd-5knww\" (UID: \"369e56ad-d701-4496-b7cb-f6d0cf4c156e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:04.089193 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:04.089157    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:04.227415 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:04.227387    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"]
Jul 17 14:59:04.230597 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:04.230573    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369e56ad_d701_4496_b7cb_f6d0cf4c156e.slice/crio-5972f5994c1e283b6a899487e870c8a3bd2f98bfc5522fee06e2fa9697cdb7a5 WatchSource:0}: Error finding container 5972f5994c1e283b6a899487e870c8a3bd2f98bfc5522fee06e2fa9697cdb7a5: Status 404 returned error can't find the container with id 5972f5994c1e283b6a899487e870c8a3bd2f98bfc5522fee06e2fa9697cdb7a5
Jul 17 14:59:05.213497 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:05.213452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww" event={"ID":"369e56ad-d701-4496-b7cb-f6d0cf4c156e","Type":"ContainerStarted","Data":"5972f5994c1e283b6a899487e870c8a3bd2f98bfc5522fee06e2fa9697cdb7a5"}
Jul 17 14:59:06.219204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:06.219110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww" event={"ID":"369e56ad-d701-4496-b7cb-f6d0cf4c156e","Type":"ContainerStarted","Data":"24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080"}
Jul 17 14:59:06.219568 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:06.219241    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:06.240866 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:06.240814    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww" podStartSLOduration=1.604226133 podStartE2EDuration="3.240794653s" totalImagesPullingTime="1.63656852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:06.240275144 +0000 UTC m=+556.807306277" watchObservedRunningTime="2026-07-17 14:59:06.240794653 +0000 UTC m=+556.807825784"
Jul 17 14:59:10.522556 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.522518    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"]
Jul 17 14:59:10.526807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.526788    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.531533 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.531514    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mwr6q\""
Jul 17 14:59:10.548862 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.548837    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"]
Jul 17 14:59:10.667454 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.667421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxckm\" (UniqueName: \"kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.667600 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.667478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.768487 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.768439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxckm\" (UniqueName: \"kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.768636 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.768503    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.768829 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.768812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.782592 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.782531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxckm\" (UniqueName: \"kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.836548 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.836514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:10.980387 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:10.980359    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"]
Jul 17 14:59:10.982318 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:10.982286    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406758db_faca_4b23_895e_910ee4b59437.slice/crio-03bb7db8bb9d9e5cdd305f968f95a13e6e090158c2e292669fabbbcc8bb209f3 WatchSource:0}: Error finding container 03bb7db8bb9d9e5cdd305f968f95a13e6e090158c2e292669fabbbcc8bb209f3: Status 404 returned error can't find the container with id 03bb7db8bb9d9e5cdd305f968f95a13e6e090158c2e292669fabbbcc8bb209f3
Jul 17 14:59:11.242429 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:11.242396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx" event={"ID":"406758db-faca-4b23-895e-910ee4b59437","Type":"ContainerStarted","Data":"03bb7db8bb9d9e5cdd305f968f95a13e6e090158c2e292669fabbbcc8bb209f3"}
Jul 17 14:59:14.209209 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:14.209179    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4vpnh"
Jul 17 14:59:16.267711 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:16.267613    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx" event={"ID":"406758db-faca-4b23-895e-910ee4b59437","Type":"ContainerStarted","Data":"bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5"}
Jul 17 14:59:16.267711 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:16.267699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:16.292205 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:16.292155    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx" podStartSLOduration=1.398923153 podStartE2EDuration="6.292137232s" totalImagesPullingTime="4.893214079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:16.289913 +0000 UTC m=+566.856944126" watchObservedRunningTime="2026-07-17 14:59:16.292137232 +0000 UTC m=+566.859168364"
Jul 17 14:59:17.226614 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:17.226579    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:18.935674 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.935636    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"]
Jul 17 14:59:18.938418 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.938398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:18.942622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.942601    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 17 14:59:18.942622 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.942614    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 17 14:59:18.942786 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.942658    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-brc6n\""
Jul 17 14:59:18.952476 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:18.951549    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"]
Jul 17 14:59:19.042116 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.042051    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be659dc6-56b6-40f4-b054-bf7432769ba2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.042301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.042180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be659dc6-56b6-40f4-b054-bf7432769ba2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.042301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.042237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwl9n\" (UniqueName: \"kubernetes.io/projected/be659dc6-56b6-40f4-b054-bf7432769ba2-kube-api-access-xwl9n\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.142853 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.142818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be659dc6-56b6-40f4-b054-bf7432769ba2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.143052 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.142892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be659dc6-56b6-40f4-b054-bf7432769ba2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.143052 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.142949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwl9n\" (UniqueName: \"kubernetes.io/projected/be659dc6-56b6-40f4-b054-bf7432769ba2-kube-api-access-xwl9n\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.143608 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.143588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/be659dc6-56b6-40f4-b054-bf7432769ba2-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.145235 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.145216    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/be659dc6-56b6-40f4-b054-bf7432769ba2-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.158472 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.158448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwl9n\" (UniqueName: \"kubernetes.io/projected/be659dc6-56b6-40f4-b054-bf7432769ba2-kube-api-access-xwl9n\") pod \"kuadrant-console-plugin-66b5774985-6zcms\" (UID: \"be659dc6-56b6-40f4-b054-bf7432769ba2\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.248977 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.248876    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"
Jul 17 14:59:19.383593 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:19.383571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms"]
Jul 17 14:59:19.385671 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:19.385640    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe659dc6_56b6_40f4_b054_bf7432769ba2.slice/crio-20c7d17c9fc698c01d04ad8dd086a5850b573070a746b4a5c4af18e3caf5c037 WatchSource:0}: Error finding container 20c7d17c9fc698c01d04ad8dd086a5850b573070a746b4a5c4af18e3caf5c037: Status 404 returned error can't find the container with id 20c7d17c9fc698c01d04ad8dd086a5850b573070a746b4a5c4af18e3caf5c037
Jul 17 14:59:20.288594 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:20.288553    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms" event={"ID":"be659dc6-56b6-40f4-b054-bf7432769ba2","Type":"ContainerStarted","Data":"20c7d17c9fc698c01d04ad8dd086a5850b573070a746b4a5c4af18e3caf5c037"}
Jul 17 14:59:27.275106 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:27.275058    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:29.071719 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.071681    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"]
Jul 17 14:59:29.072247 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.071911    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx" podUID="406758db-faca-4b23-895e-910ee4b59437" containerName="manager" containerID="cri-o://bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5" gracePeriod=2
Jul 17 14:59:29.077268 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.077232    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"]
Jul 17 14:59:29.097393 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.097362    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"]
Jul 17 14:59:29.098029 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.097676    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" containerName="manager" containerID="cri-o://24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080" gracePeriod=2
Jul 17 14:59:29.108619 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.108593    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"]
Jul 17 14:59:29.112230 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112203    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:29.112653 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112637    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" containerName="manager"
Jul 17 14:59:29.112653 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112655    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" containerName="manager"
Jul 17 14:59:29.112786 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112669    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="406758db-faca-4b23-895e-910ee4b59437" containerName="manager"
Jul 17 14:59:29.112786 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112675    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="406758db-faca-4b23-895e-910ee4b59437" containerName="manager"
Jul 17 14:59:29.112786 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112779    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="406758db-faca-4b23-895e-910ee4b59437" containerName="manager"
Jul 17 14:59:29.112943 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.112794    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" containerName="manager"
Jul 17 14:59:29.116905 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.116886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.123581 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.123544    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"369e56ad-d701-4496-b7cb-f6d0cf4c156e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-5knww\": pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:29.131986 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.131957    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.132134 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.132015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqzhq\" (UniqueName: \"kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.137018 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.136862    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:29.139961 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.139928    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:29.142148 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.141819    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:29.142613 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.142456    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:29.162587 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.162561    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"]
Jul 17 14:59:29.165451 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.165432    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:29.183864 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.183839    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"]
Jul 17 14:59:29.199131 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.199092    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:29.201862 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.201826    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:29.233009 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.232977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwc26\" (UniqueName: \"kubernetes.io/projected/5e92c85c-81a6-46d0-ac5d-941ce155a782-kube-api-access-pwc26\") pod \"limitador-operator-controller-manager-859d7b78bd-44dv5\" (UID: \"5e92c85c-81a6-46d0-ac5d-941ce155a782\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:29.233239 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.233017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.233239 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.233060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqzhq\" (UniqueName: \"kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.233491 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.233461    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.245115 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.245090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqzhq\" (UniqueName: \"kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bfmp7\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.334657 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.334557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwc26\" (UniqueName: \"kubernetes.io/projected/5e92c85c-81a6-46d0-ac5d-941ce155a782-kube-api-access-pwc26\") pod \"limitador-operator-controller-manager-859d7b78bd-44dv5\" (UID: \"5e92c85c-81a6-46d0-ac5d-941ce155a782\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:29.350194 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.350163    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwc26\" (UniqueName: \"kubernetes.io/projected/5e92c85c-81a6-46d0-ac5d-941ce155a782-kube-api-access-pwc26\") pod \"limitador-operator-controller-manager-859d7b78bd-44dv5\" (UID: \"5e92c85c-81a6-46d0-ac5d-941ce155a782\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:29.498334 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.498260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:29.507533 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:29.507507    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:29.949439 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.949397    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:29.951507 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:29.951471    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:30.065358 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.065322    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 14:59:30.073906 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.073879    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.090892 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.090865    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 14:59:30.142009 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.141975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.142195 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.142118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvvmc\" (UniqueName: \"kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.242951 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.242848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvvmc\" (UniqueName: \"kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.243136 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.242962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.243348 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.243330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.253520 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.253486    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvvmc\" (UniqueName: \"kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc\") pod \"kuadrant-operator-controller-manager-86878c769f-b4g5s\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:30.386045 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:30.386015    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:41.328173 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.327886    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.330382 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.329536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 14:59:41.330382 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.330262    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.345510 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:41.345469    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e3d4cd_0636_4ab3_bf03_5c1ce0d0efe6.slice/crio-52215c7a2f72b106b0ade4b868fe4611e09c21c1212233bbcb9c407b94797324 WatchSource:0}: Error finding container 52215c7a2f72b106b0ade4b868fe4611e09c21c1212233bbcb9c407b94797324: Status 404 returned error can't find the container with id 52215c7a2f72b106b0ade4b868fe4611e09c21c1212233bbcb9c407b94797324
Jul 17 14:59:41.354482 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.354462    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.358000 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.357676    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.360123 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.360096    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"]
Jul 17 14:59:41.361823 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.361790    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.362142 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:41.362119    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e92c85c_81a6_46d0_ac5d_941ce155a782.slice/crio-50aae59d87ca7c450351aa428469af573444a60cbd7395eaba8ff99b5db784b0 WatchSource:0}: Error finding container 50aae59d87ca7c450351aa428469af573444a60cbd7395eaba8ff99b5db784b0: Status 404 returned error can't find the container with id 50aae59d87ca7c450351aa428469af573444a60cbd7395eaba8ff99b5db784b0
Jul 17 14:59:41.381899 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.381815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:41.384899 ip-10-0-138-81 kubenswrapper[2564]: W0717 14:59:41.384872    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf4624e_a187_464e_98d4_b10601fd2514.slice/crio-90e8ad9ddc1b7e0e9fecbb01bd087ae341bdd39fc3e47dff3f473e401193a7e6 WatchSource:0}: Error finding container 90e8ad9ddc1b7e0e9fecbb01bd087ae341bdd39fc3e47dff3f473e401193a7e6: Status 404 returned error can't find the container with id 90e8ad9ddc1b7e0e9fecbb01bd087ae341bdd39fc3e47dff3f473e401193a7e6
Jul 17 14:59:41.409827 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.409797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms" event={"ID":"be659dc6-56b6-40f4-b054-bf7432769ba2","Type":"ContainerStarted","Data":"7f57e64a891b4e36b70f0ddae50c1711d5cc67470aeabf542a08d823c74c2016"}
Jul 17 14:59:41.411118 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.411094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" event={"ID":"8cf4624e-a187-464e-98d4-b10601fd2514","Type":"ContainerStarted","Data":"90e8ad9ddc1b7e0e9fecbb01bd087ae341bdd39fc3e47dff3f473e401193a7e6"}
Jul 17 14:59:41.412679 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.412658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5" event={"ID":"5e92c85c-81a6-46d0-ac5d-941ce155a782","Type":"ContainerStarted","Data":"50aae59d87ca7c450351aa428469af573444a60cbd7395eaba8ff99b5db784b0"}
Jul 17 14:59:41.414216 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.414189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" event={"ID":"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6","Type":"ContainerStarted","Data":"52215c7a2f72b106b0ade4b868fe4611e09c21c1212233bbcb9c407b94797324"}
Jul 17 14:59:41.414334 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.414250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:41.415376 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.415360    2564 generic.go:356] "Generic (PLEG): container finished" podID="406758db-faca-4b23-895e-910ee4b59437" containerID="bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5" exitCode=0
Jul 17 14:59:41.415443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.415405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.415443 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.415417    2564 scope.go:117] "RemoveContainer" containerID="bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5"
Jul 17 14:59:41.417604 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.417584    2564 generic.go:356] "Generic (PLEG): container finished" podID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" containerID="24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080" exitCode=0
Jul 17 14:59:41.417700 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.417612    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.426651 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.426628    2564 scope.go:117] "RemoveContainer" containerID="bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5"
Jul 17 14:59:41.427155 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.427129    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5\": container with ID starting with bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5 not found: ID does not exist" containerID="bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5"
Jul 17 14:59:41.427240 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.427165    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5"} err="failed to get container status \"bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5\": rpc error: code = NotFound desc = could not find container \"bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5\": container with ID starting with bbfa008bed003f9b73a54d6084a2b9aad2072eb36660e93f730ee78574f68ff5 not found: ID does not exist"
Jul 17 14:59:41.427240 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.427191    2564 scope.go:117] "RemoveContainer" containerID="24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080"
Jul 17 14:59:41.432747 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.432716    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.433314 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.433271    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6zcms" podStartSLOduration=1.592037662 podStartE2EDuration="23.43325391s" totalImagesPullingTime="21.841216248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:41.430627585 +0000 UTC m=+591.997658718" watchObservedRunningTime="2026-07-17 14:59:41.43325391 +0000 UTC m=+592.000285045"
Jul 17 14:59:41.434775 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.434745    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.436652 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.436633    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.438759 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.438741    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.441004 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.440991    2564 scope.go:117] "RemoveContainer" containerID="24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080"
Jul 17 14:59:41.441456 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.441433    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080\": container with ID starting with 24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080 not found: ID does not exist" containerID="24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080"
Jul 17 14:59:41.441606 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.441583    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080"} err="failed to get container status \"24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080\": rpc error: code = NotFound desc = could not find container \"24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080\": container with ID starting with 24ab519369aae0abdfa135604961c2078fb1d21016a0cab966ab2d398eea4080 not found: ID does not exist"
Jul 17 14:59:41.443785 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.443764    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume\") pod \"406758db-faca-4b23-895e-910ee4b59437\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") "
Jul 17 14:59:41.443875 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.443820    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxckm\" (UniqueName: \"kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm\") pod \"406758db-faca-4b23-895e-910ee4b59437\" (UID: \"406758db-faca-4b23-895e-910ee4b59437\") "
Jul 17 14:59:41.443935 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.443898    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dsbf\" (UniqueName: \"kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf\") pod \"369e56ad-d701-4496-b7cb-f6d0cf4c156e\" (UID: \"369e56ad-d701-4496-b7cb-f6d0cf4c156e\") "
Jul 17 14:59:41.444463 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.444433    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "406758db-faca-4b23-895e-910ee4b59437" (UID: "406758db-faca-4b23-895e-910ee4b59437"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:59:41.446419 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.446329    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm" (OuterVolumeSpecName: "kube-api-access-zxckm") pod "406758db-faca-4b23-895e-910ee4b59437" (UID: "406758db-faca-4b23-895e-910ee4b59437"). InnerVolumeSpecName "kube-api-access-zxckm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:59:41.446419 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.446398    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf" (OuterVolumeSpecName: "kube-api-access-2dsbf") pod "369e56ad-d701-4496-b7cb-f6d0cf4c156e" (UID: "369e56ad-d701-4496-b7cb-f6d0cf4c156e"). InnerVolumeSpecName "kube-api-access-2dsbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:59:41.464140 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.464091    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" podStartSLOduration=11.464058867 podStartE2EDuration="11.464058867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:41.462680319 +0000 UTC m=+592.029711453" watchObservedRunningTime="2026-07-17 14:59:41.464058867 +0000 UTC m=+592.031090000"
Jul 17 14:59:41.545036 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.544992    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dsbf\" (UniqueName: \"kubernetes.io/projected/369e56ad-d701-4496-b7cb-f6d0cf4c156e-kube-api-access-2dsbf\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:59:41.545301 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.545280    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/406758db-faca-4b23-895e-910ee4b59437-extensions-socket-volume\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:59:41.545426 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.545412    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxckm\" (UniqueName: \"kubernetes.io/projected/406758db-faca-4b23-895e-910ee4b59437-kube-api-access-zxckm\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:59:41.730762 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.730725    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.732835 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.732811    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.734779 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.734757    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5knww\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5knww"
Jul 17 14:59:41.736717 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:41.736699    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4fzjx\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="406758db-faca-4b23-895e-910ee4b59437" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4fzjx"
Jul 17 14:59:41.949585 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.949543    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="369e56ad-d701-4496-b7cb-f6d0cf4c156e" path="/var/lib/kubelet/pods/369e56ad-d701-4496-b7cb-f6d0cf4c156e/volumes"
Jul 17 14:59:41.949880 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:41.949868    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="406758db-faca-4b23-895e-910ee4b59437" path="/var/lib/kubelet/pods/406758db-faca-4b23-895e-910ee4b59437/volumes"
Jul 17 14:59:42.428329 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.428228    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5" event={"ID":"5e92c85c-81a6-46d0-ac5d-941ce155a782","Type":"ContainerStarted","Data":"091c2482144f79e84bf57c998def8707b8a266c2530afdd58c2677436ab0dc0e"}
Jul 17 14:59:42.428765 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.428365    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:42.429788 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.429763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" event={"ID":"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6","Type":"ContainerStarted","Data":"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"}
Jul 17 14:59:42.433090 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.433047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" event={"ID":"8cf4624e-a187-464e-98d4-b10601fd2514","Type":"ContainerStarted","Data":"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"}
Jul 17 14:59:42.433204 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.433186    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:42.449807 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.449755    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5" podStartSLOduration=13.449743528 podStartE2EDuration="13.449743528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:42.448858715 +0000 UTC m=+593.015889848" watchObservedRunningTime="2026-07-17 14:59:42.449743528 +0000 UTC m=+593.016774660"
Jul 17 14:59:42.473633 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:42.473590    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" podStartSLOduration=13.473578069 podStartE2EDuration="13.473578069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 14:59:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 14:59:42.471567299 +0000 UTC m=+593.038598423" watchObservedRunningTime="2026-07-17 14:59:42.473578069 +0000 UTC m=+593.040609201"
Jul 17 14:59:49.853116 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:49.853088    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:59:49.858508 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:49.858486    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:59:49.860313 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:49.860293    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 14:59:49.865560 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:49.865545    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 14:59:53.440132 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.440099    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:53.440609 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.440266    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 14:59:53.440609 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.440303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-44dv5"
Jul 17 14:59:53.577640 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.577603    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:53.577866 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.577812    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" podUID="8cf4624e-a187-464e-98d4-b10601fd2514" containerName="manager" containerID="cri-o://3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2" gracePeriod=10
Jul 17 14:59:53.835869 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.835846    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:53.947025 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.946939    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqzhq\" (UniqueName: \"kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq\") pod \"8cf4624e-a187-464e-98d4-b10601fd2514\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") "
Jul 17 14:59:53.947025 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.947000    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume\") pod \"8cf4624e-a187-464e-98d4-b10601fd2514\" (UID: \"8cf4624e-a187-464e-98d4-b10601fd2514\") "
Jul 17 14:59:53.947396 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.947368    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8cf4624e-a187-464e-98d4-b10601fd2514" (UID: "8cf4624e-a187-464e-98d4-b10601fd2514"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 14:59:53.949160 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:53.949134    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq" (OuterVolumeSpecName: "kube-api-access-hqzhq") pod "8cf4624e-a187-464e-98d4-b10601fd2514" (UID: "8cf4624e-a187-464e-98d4-b10601fd2514"). InnerVolumeSpecName "kube-api-access-hqzhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 14:59:54.048625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.048591    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8cf4624e-a187-464e-98d4-b10601fd2514-extensions-socket-volume\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:59:54.048625 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.048619    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqzhq\" (UniqueName: \"kubernetes.io/projected/8cf4624e-a187-464e-98d4-b10601fd2514-kube-api-access-hqzhq\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 14:59:54.495441 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.495402    2564 generic.go:356] "Generic (PLEG): container finished" podID="8cf4624e-a187-464e-98d4-b10601fd2514" containerID="3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2" exitCode=0
Jul 17 14:59:54.495931 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.495450    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"
Jul 17 14:59:54.495931 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.495492    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" event={"ID":"8cf4624e-a187-464e-98d4-b10601fd2514","Type":"ContainerDied","Data":"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"}
Jul 17 14:59:54.495931 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.495561    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7" event={"ID":"8cf4624e-a187-464e-98d4-b10601fd2514","Type":"ContainerDied","Data":"90e8ad9ddc1b7e0e9fecbb01bd087ae341bdd39fc3e47dff3f473e401193a7e6"}
Jul 17 14:59:54.495931 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.495601    2564 scope.go:117] "RemoveContainer" containerID="3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"
Jul 17 14:59:54.504193 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.504176    2564 scope.go:117] "RemoveContainer" containerID="3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"
Jul 17 14:59:54.504425 ip-10-0-138-81 kubenswrapper[2564]: E0717 14:59:54.504409    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2\": container with ID starting with 3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2 not found: ID does not exist" containerID="3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"
Jul 17 14:59:54.504479 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.504434    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2"} err="failed to get container status \"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2\": rpc error: code = NotFound desc = could not find container \"3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2\": container with ID starting with 3597c703b11c3f46486568633a7d757a5c36b4456e00ff2359bc07cca3cb8bd2 not found: ID does not exist"
Jul 17 14:59:54.524671 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.524653    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:54.534408 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:54.534388    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bfmp7"]
Jul 17 14:59:55.949867 ip-10-0-138-81 kubenswrapper[2564]: I0717 14:59:55.949830    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf4624e-a187-464e-98d4-b10601fd2514" path="/var/lib/kubelet/pods/8cf4624e-a187-464e-98d4-b10601fd2514/volumes"
Jul 17 15:00:13.192951 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.192866    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:00:13.193419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.193238    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf4624e-a187-464e-98d4-b10601fd2514" containerName="manager"
Jul 17 15:00:13.193419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.193251    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf4624e-a187-464e-98d4-b10601fd2514" containerName="manager"
Jul 17 15:00:13.193419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.193324    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf4624e-a187-464e-98d4-b10601fd2514" containerName="manager"
Jul 17 15:00:13.200978 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.200958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.204218 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.204188    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 15:00:13.206472 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.206450    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:00:13.246814 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.246778    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:00:13.319698 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.319665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-config-file\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.319870 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.319734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mswsf\" (UniqueName: \"kubernetes.io/projected/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-kube-api-access-mswsf\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.420593 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.420550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mswsf\" (UniqueName: \"kubernetes.io/projected/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-kube-api-access-mswsf\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.420799 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.420624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-config-file\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.421271 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.421251    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-config-file\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.430645 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.430623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mswsf\" (UniqueName: \"kubernetes.io/projected/21f66bd4-4c7c-458a-83a3-d0e4af585e4b-kube-api-access-mswsf\") pod \"limitador-limitador-54d6d988d5-bvnbh\" (UID: \"21f66bd4-4c7c-458a-83a3-d0e4af585e4b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.513100 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.512994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:13.646481 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.646448    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:00:13.650172 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:00:13.650134    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f66bd4_4c7c_458a_83a3_d0e4af585e4b.slice/crio-3d03ab3dab3dec71e07f966bb9a7411a733224ab6806d862074810d6b4974429 WatchSource:0}: Error finding container 3d03ab3dab3dec71e07f966bb9a7411a733224ab6806d862074810d6b4974429: Status 404 returned error can't find the container with id 3d03ab3dab3dec71e07f966bb9a7411a733224ab6806d862074810d6b4974429
Jul 17 15:00:13.911486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.911455    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:13.914876 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.914860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:13.917738 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.917718    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z5dcn\""
Jul 17 15:00:13.924893 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:13.924872    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:14.026194 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.026160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2t67\" (UniqueName: \"kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67\") pod \"authorino-75d556bd77-v558d\" (UID: \"459c2b95-96a2-4f82-874d-8eb93ef99db0\") " pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:14.126879 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.126846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2t67\" (UniqueName: \"kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67\") pod \"authorino-75d556bd77-v558d\" (UID: \"459c2b95-96a2-4f82-874d-8eb93ef99db0\") " pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:14.137363 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.137341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2t67\" (UniqueName: \"kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67\") pod \"authorino-75d556bd77-v558d\" (UID: \"459c2b95-96a2-4f82-874d-8eb93ef99db0\") " pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:14.224625 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.224550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:14.381095 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.381033    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:14.385155 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:00:14.385126    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod459c2b95_96a2_4f82_874d_8eb93ef99db0.slice/crio-20997463bcb4e011a97541159022952f24a0948074e8d8db572fd37b310a1838 WatchSource:0}: Error finding container 20997463bcb4e011a97541159022952f24a0948074e8d8db572fd37b310a1838: Status 404 returned error can't find the container with id 20997463bcb4e011a97541159022952f24a0948074e8d8db572fd37b310a1838
Jul 17 15:00:14.597123 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.597058    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v558d" event={"ID":"459c2b95-96a2-4f82-874d-8eb93ef99db0","Type":"ContainerStarted","Data":"20997463bcb4e011a97541159022952f24a0948074e8d8db572fd37b310a1838"}
Jul 17 15:00:14.598762 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:14.598733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh" event={"ID":"21f66bd4-4c7c-458a-83a3-d0e4af585e4b","Type":"ContainerStarted","Data":"3d03ab3dab3dec71e07f966bb9a7411a733224ab6806d862074810d6b4974429"}
Jul 17 15:00:16.613740 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:16.613697    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh" event={"ID":"21f66bd4-4c7c-458a-83a3-d0e4af585e4b","Type":"ContainerStarted","Data":"35988801f285fa95d03d7d65c8b6bfd7f5c50bc6fd4ae70be0161e1f42c2972c"}
Jul 17 15:00:16.614252 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:16.613957    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:16.636949 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:16.636886    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh" podStartSLOduration=0.841482556 podStartE2EDuration="3.636864213s" totalImagesPullingTime="2.795381657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:00:16.633794007 +0000 UTC m=+627.200825141" watchObservedRunningTime="2026-07-17 15:00:16.636864213 +0000 UTC m=+627.203895346"
Jul 17 15:00:19.633184 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:19.633147    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v558d" event={"ID":"459c2b95-96a2-4f82-874d-8eb93ef99db0","Type":"ContainerStarted","Data":"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"}
Jul 17 15:00:19.653861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:19.653810    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-v558d" podStartSLOduration=2.317592061 podStartE2EDuration="6.653793045s" totalImagesPullingTime="4.336200984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:00:19.651762785 +0000 UTC m=+630.218793916" watchObservedRunningTime="2026-07-17 15:00:19.653793045 +0000 UTC m=+630.220824176"
Jul 17 15:00:27.622851 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:27.622818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"
Jul 17 15:00:41.542916 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.542881    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:41.546222 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.546205    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:41.557823 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.557798    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:41.650486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.650456    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p99d\" (UniqueName: \"kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d\") pod \"authorino-5d4b67c446-j7v5b\" (UID: \"62552f3a-26d3-4eb5-ac48-ba1af82de803\") " pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:41.751889 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.751840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p99d\" (UniqueName: \"kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d\") pod \"authorino-5d4b67c446-j7v5b\" (UID: \"62552f3a-26d3-4eb5-ac48-ba1af82de803\") " pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:41.763886 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.763855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p99d\" (UniqueName: \"kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d\") pod \"authorino-5d4b67c446-j7v5b\" (UID: \"62552f3a-26d3-4eb5-ac48-ba1af82de803\") " pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:41.782671 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.782645    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:41.782912 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.782896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:41.822735 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.822703    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:41.826153 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.826132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:41.838781 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.838754    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:41.930198 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.930174    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:41.932295 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:00:41.932272    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62552f3a_26d3_4eb5_ac48_ba1af82de803.slice/crio-c7251a0c8ce333cbaf6c3574e2024786abae2978277ebcc7056961a8731d5a7f WatchSource:0}: Error finding container c7251a0c8ce333cbaf6c3574e2024786abae2978277ebcc7056961a8731d5a7f: Status 404 returned error can't find the container with id c7251a0c8ce333cbaf6c3574e2024786abae2978277ebcc7056961a8731d5a7f
Jul 17 15:00:41.954569 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:41.954541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkt94\" (UniqueName: \"kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94\") pod \"authorino-7d5f66d6b9-ftz7l\" (UID: \"8fbc754e-2f3b-423b-822c-062ed1846e79\") " pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:42.055770 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.055736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkt94\" (UniqueName: \"kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94\") pod \"authorino-7d5f66d6b9-ftz7l\" (UID: \"8fbc754e-2f3b-423b-822c-062ed1846e79\") " pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:42.069516 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.069489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkt94\" (UniqueName: \"kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94\") pod \"authorino-7d5f66d6b9-ftz7l\" (UID: \"8fbc754e-2f3b-423b-822c-062ed1846e79\") " pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:42.139954 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.139882    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:42.149790 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.149759    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:42.198305 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.198273    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:00:42.202163 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.202140    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.207575 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.207553    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 17 15:00:42.218798 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.218769    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:00:42.257634 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.257602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxttc\" (UniqueName: \"kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.257812 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.257673    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.288640 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.288475    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:42.290672 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:00:42.290637    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fbc754e_2f3b_423b_822c_062ed1846e79.slice/crio-3a37713912b2aec0273fba0a93d91d4994c1f1452e863bb26a60212f28ad79e4 WatchSource:0}: Error finding container 3a37713912b2aec0273fba0a93d91d4994c1f1452e863bb26a60212f28ad79e4: Status 404 returned error can't find the container with id 3a37713912b2aec0273fba0a93d91d4994c1f1452e863bb26a60212f28ad79e4
Jul 17 15:00:42.358129 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.358084    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxttc\" (UniqueName: \"kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.358282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.358149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.360427 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.360408    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.368512 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.368489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxttc\" (UniqueName: \"kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc\") pod \"authorino-7794b47dd9-z2tdc\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") " pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.512844 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.512770    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:00:42.678277 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.678053    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:00:42.680480 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:00:42.680456    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70c33948_ea3c_44b4_a0bb_5f0ea8a17aca.slice/crio-1746d1925cf8cfb815e56e2fa3fe9e8bd480be5a2181fed5662b0d1ae5173eb7 WatchSource:0}: Error finding container 1746d1925cf8cfb815e56e2fa3fe9e8bd480be5a2181fed5662b0d1ae5173eb7: Status 404 returned error can't find the container with id 1746d1925cf8cfb815e56e2fa3fe9e8bd480be5a2181fed5662b0d1ae5173eb7
Jul 17 15:00:42.739451 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.739423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" event={"ID":"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca","Type":"ContainerStarted","Data":"1746d1925cf8cfb815e56e2fa3fe9e8bd480be5a2181fed5662b0d1ae5173eb7"}
Jul 17 15:00:42.740608 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.740585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" event={"ID":"8fbc754e-2f3b-423b-822c-062ed1846e79","Type":"ContainerStarted","Data":"3a37713912b2aec0273fba0a93d91d4994c1f1452e863bb26a60212f28ad79e4"}
Jul 17 15:00:42.741672 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:42.741644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" event={"ID":"62552f3a-26d3-4eb5-ac48-ba1af82de803","Type":"ContainerStarted","Data":"c7251a0c8ce333cbaf6c3574e2024786abae2978277ebcc7056961a8731d5a7f"}
Jul 17 15:00:43.747499 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:43.747433    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" event={"ID":"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca","Type":"ContainerStarted","Data":"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"}
Jul 17 15:00:43.777761 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:43.777698    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" podStartSLOduration=1.213849772 podStartE2EDuration="1.777675214s" totalImagesPullingTime="563.825442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:00:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:00:43.774894718 +0000 UTC m=+654.341925871" watchObservedRunningTime="2026-07-17 15:00:43.777675214 +0000 UTC m=+654.344706347"
Jul 17 15:00:43.811166 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:43.810481    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:43.811166 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:43.810680    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-v558d" podUID="459c2b95-96a2-4f82-874d-8eb93ef99db0" containerName="authorino" containerID="cri-o://271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd" gracePeriod=30
Jul 17 15:00:44.072148 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.072123    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:44.173848 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.173816    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2t67\" (UniqueName: \"kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67\") pod \"459c2b95-96a2-4f82-874d-8eb93ef99db0\" (UID: \"459c2b95-96a2-4f82-874d-8eb93ef99db0\") "
Jul 17 15:00:44.175874 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.175838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67" (OuterVolumeSpecName: "kube-api-access-w2t67") pod "459c2b95-96a2-4f82-874d-8eb93ef99db0" (UID: "459c2b95-96a2-4f82-874d-8eb93ef99db0"). InnerVolumeSpecName "kube-api-access-w2t67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:00:44.274960 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.274890    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2t67\" (UniqueName: \"kubernetes.io/projected/459c2b95-96a2-4f82-874d-8eb93ef99db0-kube-api-access-w2t67\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:00:44.753953 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.753918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" event={"ID":"8fbc754e-2f3b-423b-822c-062ed1846e79","Type":"ContainerStarted","Data":"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"}
Jul 17 15:00:44.754435 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.754002    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" podUID="8fbc754e-2f3b-423b-822c-062ed1846e79" containerName="authorino" containerID="cri-o://632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00" gracePeriod=30
Jul 17 15:00:44.755249 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.755225    2564 generic.go:356] "Generic (PLEG): container finished" podID="459c2b95-96a2-4f82-874d-8eb93ef99db0" containerID="271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd" exitCode=0
Jul 17 15:00:44.755318 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.755281    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v558d"
Jul 17 15:00:44.755382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.755310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v558d" event={"ID":"459c2b95-96a2-4f82-874d-8eb93ef99db0","Type":"ContainerDied","Data":"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"}
Jul 17 15:00:44.755382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.755348    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v558d" event={"ID":"459c2b95-96a2-4f82-874d-8eb93ef99db0","Type":"ContainerDied","Data":"20997463bcb4e011a97541159022952f24a0948074e8d8db572fd37b310a1838"}
Jul 17 15:00:44.755382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.755372    2564 scope.go:117] "RemoveContainer" containerID="271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"
Jul 17 15:00:44.757711 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.757466    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" podUID="62552f3a-26d3-4eb5-ac48-ba1af82de803" containerName="authorino" containerID="cri-o://0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3" gracePeriod=30
Jul 17 15:00:44.757711 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.757611    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" event={"ID":"62552f3a-26d3-4eb5-ac48-ba1af82de803","Type":"ContainerStarted","Data":"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"}
Jul 17 15:00:44.767694 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.767670    2564 scope.go:117] "RemoveContainer" containerID="271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"
Jul 17 15:00:44.767980 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:00:44.767956    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd\": container with ID starting with 271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd not found: ID does not exist" containerID="271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"
Jul 17 15:00:44.768092 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.767991    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd"} err="failed to get container status \"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd\": rpc error: code = NotFound desc = could not find container \"271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd\": container with ID starting with 271acc97d025b0abc4851ea1d08fe12b95ff0103c8696a3d4c3a7b7f92c5c1cd not found: ID does not exist"
Jul 17 15:00:44.777948 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.777901    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" podStartSLOduration=2.253832669 podStartE2EDuration="3.77788931s" totalImagesPullingTime="1.524056641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:00:44.777254948 +0000 UTC m=+655.344286094" watchObservedRunningTime="2026-07-17 15:00:44.77788931 +0000 UTC m=+655.344920441"
Jul 17 15:00:44.799015 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.798971    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" podStartSLOduration=2.03567076 podStartE2EDuration="3.798957866s" totalImagesPullingTime="1.763287106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:00:44.797748537 +0000 UTC m=+655.364779669" watchObservedRunningTime="2026-07-17 15:00:44.798957866 +0000 UTC m=+655.365988998"
Jul 17 15:00:44.818521 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.818493    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:44.822866 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:44.822839    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v558d"]
Jul 17 15:00:45.035035 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.035015    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:45.036351 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.036336    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:45.082431 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.082407    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkt94\" (UniqueName: \"kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94\") pod \"8fbc754e-2f3b-423b-822c-062ed1846e79\" (UID: \"8fbc754e-2f3b-423b-822c-062ed1846e79\") "
Jul 17 15:00:45.082431 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.082443    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p99d\" (UniqueName: \"kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d\") pod \"62552f3a-26d3-4eb5-ac48-ba1af82de803\" (UID: \"62552f3a-26d3-4eb5-ac48-ba1af82de803\") "
Jul 17 15:00:45.084696 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.084663    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d" (OuterVolumeSpecName: "kube-api-access-5p99d") pod "62552f3a-26d3-4eb5-ac48-ba1af82de803" (UID: "62552f3a-26d3-4eb5-ac48-ba1af82de803"). InnerVolumeSpecName "kube-api-access-5p99d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:00:45.084794 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.084738    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94" (OuterVolumeSpecName: "kube-api-access-jkt94") pod "8fbc754e-2f3b-423b-822c-062ed1846e79" (UID: "8fbc754e-2f3b-423b-822c-062ed1846e79"). InnerVolumeSpecName "kube-api-access-jkt94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:00:45.183496 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.183450    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkt94\" (UniqueName: \"kubernetes.io/projected/8fbc754e-2f3b-423b-822c-062ed1846e79-kube-api-access-jkt94\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:00:45.183496 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.183489    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p99d\" (UniqueName: \"kubernetes.io/projected/62552f3a-26d3-4eb5-ac48-ba1af82de803-kube-api-access-5p99d\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:00:45.763058 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.763022    2564 generic.go:356] "Generic (PLEG): container finished" podID="8fbc754e-2f3b-423b-822c-062ed1846e79" containerID="632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00" exitCode=0
Jul 17 15:00:45.763517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.763086    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l"
Jul 17 15:00:45.763517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.763118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" event={"ID":"8fbc754e-2f3b-423b-822c-062ed1846e79","Type":"ContainerDied","Data":"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"}
Jul 17 15:00:45.763517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.763164    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d5f66d6b9-ftz7l" event={"ID":"8fbc754e-2f3b-423b-822c-062ed1846e79","Type":"ContainerDied","Data":"3a37713912b2aec0273fba0a93d91d4994c1f1452e863bb26a60212f28ad79e4"}
Jul 17 15:00:45.763517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.763181    2564 scope.go:117] "RemoveContainer" containerID="632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"
Jul 17 15:00:45.765532 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.765505    2564 generic.go:356] "Generic (PLEG): container finished" podID="62552f3a-26d3-4eb5-ac48-ba1af82de803" containerID="0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3" exitCode=0
Jul 17 15:00:45.765648 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.765552    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-j7v5b"
Jul 17 15:00:45.765648 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.765557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" event={"ID":"62552f3a-26d3-4eb5-ac48-ba1af82de803","Type":"ContainerDied","Data":"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"}
Jul 17 15:00:45.765648 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.765579    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-j7v5b" event={"ID":"62552f3a-26d3-4eb5-ac48-ba1af82de803","Type":"ContainerDied","Data":"c7251a0c8ce333cbaf6c3574e2024786abae2978277ebcc7056961a8731d5a7f"}
Jul 17 15:00:45.773627 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.773606    2564 scope.go:117] "RemoveContainer" containerID="632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"
Jul 17 15:00:45.773984 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:00:45.773956    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00\": container with ID starting with 632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00 not found: ID does not exist" containerID="632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"
Jul 17 15:00:45.774088 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.773991    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00"} err="failed to get container status \"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00\": rpc error: code = NotFound desc = could not find container \"632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00\": container with ID starting with 632b3b4a7cbe5d260f81b09cde619715708ee5303f0faebc3343c5c277d98f00 not found: ID does not exist"
Jul 17 15:00:45.774088 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.774010    2564 scope.go:117] "RemoveContainer" containerID="0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"
Jul 17 15:00:45.783868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.783848    2564 scope.go:117] "RemoveContainer" containerID="0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"
Jul 17 15:00:45.784345 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:00:45.784318    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3\": container with ID starting with 0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3 not found: ID does not exist" containerID="0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"
Jul 17 15:00:45.784439 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.784353    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3"} err="failed to get container status \"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3\": rpc error: code = NotFound desc = could not find container \"0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3\": container with ID starting with 0ce56f3d3dec2f36f97f8688f6a677bc22ba313d3ca9d6033bd2fba03bc99eb3 not found: ID does not exist"
Jul 17 15:00:45.789555 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.789534    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:45.791927 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.791904    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d5f66d6b9-ftz7l"]
Jul 17 15:00:45.806677 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.806658    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:45.813152 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.813133    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-j7v5b"]
Jul 17 15:00:45.950641 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.950608    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="459c2b95-96a2-4f82-874d-8eb93ef99db0" path="/var/lib/kubelet/pods/459c2b95-96a2-4f82-874d-8eb93ef99db0/volumes"
Jul 17 15:00:45.950953 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.950938    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62552f3a-26d3-4eb5-ac48-ba1af82de803" path="/var/lib/kubelet/pods/62552f3a-26d3-4eb5-ac48-ba1af82de803/volumes"
Jul 17 15:00:45.951247 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:00:45.951234    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fbc754e-2f3b-423b-822c-062ed1846e79" path="/var/lib/kubelet/pods/8fbc754e-2f3b-423b-822c-062ed1846e79/volumes"
Jul 17 15:01:24.552027 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:01:24.551987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:01:57.097388 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:01:57.097351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:02:10.822832 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.822794    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"]
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823166    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fbc754e-2f3b-423b-822c-062ed1846e79" containerName="authorino"
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823177    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbc754e-2f3b-423b-822c-062ed1846e79" containerName="authorino"
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823188    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="459c2b95-96a2-4f82-874d-8eb93ef99db0" containerName="authorino"
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823195    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="459c2b95-96a2-4f82-874d-8eb93ef99db0" containerName="authorino"
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823209    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62552f3a-26d3-4eb5-ac48-ba1af82de803" containerName="authorino"
Jul 17 15:02:10.823223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823214    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="62552f3a-26d3-4eb5-ac48-ba1af82de803" containerName="authorino"
Jul 17 15:02:10.823413 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823282    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fbc754e-2f3b-423b-822c-062ed1846e79" containerName="authorino"
Jul 17 15:02:10.823413 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823292    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="62552f3a-26d3-4eb5-ac48-ba1af82de803" containerName="authorino"
Jul 17 15:02:10.823413 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.823299    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="459c2b95-96a2-4f82-874d-8eb93ef99db0" containerName="authorino"
Jul 17 15:02:10.825607 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.825589    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.830528 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.830505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qmdn8\""
Jul 17 15:02:10.830645 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.830537    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 15:02:10.830645 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.830587    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 15:02:10.830645 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.830595    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 15:02:10.848945 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.848923    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"]
Jul 17 15:02:10.906365 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906330    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.906365 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.906586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbnk\" (UniqueName: \"kubernetes.io/projected/5fc67c78-9e12-43c0-a55d-a5614dea757d-kube-api-access-brbnk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.906586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.906586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:10.906586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:10.906524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc67c78-9e12-43c0-a55d-a5614dea757d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007681 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbnk\" (UniqueName: \"kubernetes.io/projected/5fc67c78-9e12-43c0-a55d-a5614dea757d-kube-api-access-brbnk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007681 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007680    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007938 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007713    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007938 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007742    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc67c78-9e12-43c0-a55d-a5614dea757d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007938 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007814    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.007938 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.007837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.008218 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.008170    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.008278 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.008259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.008319 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.008305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.010198 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.010176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc67c78-9e12-43c0-a55d-a5614dea757d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.010319 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.010275    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc67c78-9e12-43c0-a55d-a5614dea757d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.029114 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.029092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbnk\" (UniqueName: \"kubernetes.io/projected/5fc67c78-9e12-43c0-a55d-a5614dea757d-kube-api-access-brbnk\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nkhq8\" (UID: \"5fc67c78-9e12-43c0-a55d-a5614dea757d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.136052 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.135966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:11.286803 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.286772    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"]
Jul 17 15:02:11.289796 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:02:11.289759    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc67c78_9e12_43c0_a55d_a5614dea757d.slice/crio-6145b8354c2873f711c528748a6c7953b1547708ec15fa0aed20e4b2eec90aba WatchSource:0}: Error finding container 6145b8354c2873f711c528748a6c7953b1547708ec15fa0aed20e4b2eec90aba: Status 404 returned error can't find the container with id 6145b8354c2873f711c528748a6c7953b1547708ec15fa0aed20e4b2eec90aba
Jul 17 15:02:11.291056 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.291040    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:02:11.694905 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:11.694871    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:02:12.196665 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:12.196620    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8" event={"ID":"5fc67c78-9e12-43c0-a55d-a5614dea757d","Type":"ContainerStarted","Data":"6145b8354c2873f711c528748a6c7953b1547708ec15fa0aed20e4b2eec90aba"}
Jul 17 15:02:14.209630 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:14.209598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8" event={"ID":"5fc67c78-9e12-43c0-a55d-a5614dea757d","Type":"ContainerStarted","Data":"ea0f953411ce1e652d938865668bee77d24edd99c17ea042ba91779483df97e4"}
Jul 17 15:02:14.233182 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:14.233131    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8" podStartSLOduration=1.78094762 podStartE2EDuration="4.233114774s" totalImagesPullingTime="2.452167154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:02:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:02:14.231425804 +0000 UTC m=+744.798456955" watchObservedRunningTime="2026-07-17 15:02:14.233114774 +0000 UTC m=+744.800145897"
Jul 17 15:02:21.136245 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:21.136207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:21.136696 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:21.136266    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:21.148926 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:21.148897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:21.254970 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:21.254941    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nkhq8"
Jul 17 15:02:25.496511 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:25.496479    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:02:33.027687 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:33.027651    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:02:48.304365 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:02:48.304325    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:03:03.304380 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:03:03.304341    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:04:16.112977 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.112943    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84c6465864-kx76v"]
Jul 17 15:04:16.115499 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.115484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.127404 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.127383    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84c6465864-kx76v"]
Jul 17 15:04:16.167382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.167354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1846718a-89e4-47ff-a25e-b29f73e157a2-tls-cert\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.167529 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.167417    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kt9f\" (UniqueName: \"kubernetes.io/projected/1846718a-89e4-47ff-a25e-b29f73e157a2-kube-api-access-6kt9f\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.268783 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.268749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kt9f\" (UniqueName: \"kubernetes.io/projected/1846718a-89e4-47ff-a25e-b29f73e157a2-kube-api-access-6kt9f\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.268935 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.268816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1846718a-89e4-47ff-a25e-b29f73e157a2-tls-cert\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.271210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.271189    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1846718a-89e4-47ff-a25e-b29f73e157a2-tls-cert\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.279782 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.279762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kt9f\" (UniqueName: \"kubernetes.io/projected/1846718a-89e4-47ff-a25e-b29f73e157a2-kube-api-access-6kt9f\") pod \"authorino-84c6465864-kx76v\" (UID: \"1846718a-89e4-47ff-a25e-b29f73e157a2\") " pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.424862 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.424783    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84c6465864-kx76v"
Jul 17 15:04:16.574715 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:04:16.574690    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1846718a_89e4_47ff_a25e_b29f73e157a2.slice/crio-e0deb4b1f3871d9cdb1c9fe0a7920f6c0a3a74eb104425ec3ae0c60fdc1b8996 WatchSource:0}: Error finding container e0deb4b1f3871d9cdb1c9fe0a7920f6c0a3a74eb104425ec3ae0c60fdc1b8996: Status 404 returned error can't find the container with id e0deb4b1f3871d9cdb1c9fe0a7920f6c0a3a74eb104425ec3ae0c60fdc1b8996
Jul 17 15:04:16.574813 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.574755    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84c6465864-kx76v"]
Jul 17 15:04:16.808872 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:16.808790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84c6465864-kx76v" event={"ID":"1846718a-89e4-47ff-a25e-b29f73e157a2","Type":"ContainerStarted","Data":"e0deb4b1f3871d9cdb1c9fe0a7920f6c0a3a74eb104425ec3ae0c60fdc1b8996"}
Jul 17 15:04:17.817775 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:17.817715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84c6465864-kx76v" event={"ID":"1846718a-89e4-47ff-a25e-b29f73e157a2","Type":"ContainerStarted","Data":"6643b504efe23651032d59d3ee802ab2c0eefb845f2e216beb647f938a8fd87d"}
Jul 17 15:04:17.851492 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:17.851435    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84c6465864-kx76v" podStartSLOduration=1.451425573 podStartE2EDuration="1.851413579s" totalImagesPullingTime="399.988006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:04:17.847659545 +0000 UTC m=+868.414690676" watchObservedRunningTime="2026-07-17 15:04:17.851413579 +0000 UTC m=+868.418444710"
Jul 17 15:04:17.892518 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:17.892479    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:04:17.892788 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:17.892745    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" podUID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" containerName="authorino" containerID="cri-o://8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565" gracePeriod=30
Jul 17 15:04:18.156378 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.156354    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:04:18.184999 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.184871    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxttc\" (UniqueName: \"kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc\") pod \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") "
Jul 17 15:04:18.184999 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.184967    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert\") pod \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\" (UID: \"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca\") "
Jul 17 15:04:18.187834 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.187799    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc" (OuterVolumeSpecName: "kube-api-access-mxttc") pod "70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" (UID: "70c33948-ea3c-44b4-a0bb-5f0ea8a17aca"). InnerVolumeSpecName "kube-api-access-mxttc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:04:18.197728 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.197702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" (UID: "70c33948-ea3c-44b4-a0bb-5f0ea8a17aca"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:04:18.285694 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.285657    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxttc\" (UniqueName: \"kubernetes.io/projected/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-kube-api-access-mxttc\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:04:18.285694 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.285694    2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca-tls-cert\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:04:18.824645 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.824609    2564 generic.go:356] "Generic (PLEG): container finished" podID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" containerID="8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565" exitCode=0
Jul 17 15:04:18.825131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.824674    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7794b47dd9-z2tdc"
Jul 17 15:04:18.825131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.824711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" event={"ID":"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca","Type":"ContainerDied","Data":"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"}
Jul 17 15:04:18.825131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.824759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7794b47dd9-z2tdc" event={"ID":"70c33948-ea3c-44b4-a0bb-5f0ea8a17aca","Type":"ContainerDied","Data":"1746d1925cf8cfb815e56e2fa3fe9e8bd480be5a2181fed5662b0d1ae5173eb7"}
Jul 17 15:04:18.825131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.824778    2564 scope.go:117] "RemoveContainer" containerID="8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"
Jul 17 15:04:18.835174 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.835131    2564 scope.go:117] "RemoveContainer" containerID="8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"
Jul 17 15:04:18.835485 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:04:18.835457    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565\": container with ID starting with 8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565 not found: ID does not exist" containerID="8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"
Jul 17 15:04:18.835585 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.835492    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565"} err="failed to get container status \"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565\": rpc error: code = NotFound desc = could not find container \"8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565\": container with ID starting with 8c6f2f6d9766b877b65f1d6e866c997cfa31374bc77ad3163b5e941df097e565 not found: ID does not exist"
Jul 17 15:04:18.854269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.854241    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:04:18.862407 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:18.862381    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7794b47dd9-z2tdc"]
Jul 17 15:04:19.950388 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:19.950350    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" path="/var/lib/kubelet/pods/70c33948-ea3c-44b4-a0bb-5f0ea8a17aca/volumes"
Jul 17 15:04:49.012648 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:49.012616    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:04:49.888129 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:49.888093    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:04:49.896538 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:49.896517    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:04:49.897749 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:49.897730    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:04:49.902723 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:49.902706    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:04:59.207459 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:04:59.207425    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:05:09.217100 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:05:09.217045    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:06:49.582892 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:06:49.582851    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:06:59.093294 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:06:59.093250    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:07:30.890110 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:07:30.890075    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:07:40.246483 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:07:40.246449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:07:50.393197 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:07:50.393161    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:07:59.590391 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:07:59.590355    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:08:29.585699 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:08:29.585664    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:08:39.698254 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:08:39.698215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:09:01.993378 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:01.993343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:09:12.824481 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:12.824405    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:09:38.091963 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:38.091926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:09:49.924715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:49.924668    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:09:49.927805 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:49.927785    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:09:49.930920 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:49.930896    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:09:49.934545 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:49.934527    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:09:54.404207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:09:54.404170    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:10:06.689128 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:10:06.689090    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:10:16.905882 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:10:16.905841    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:10:26.923754 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:10:26.923711    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:10:37.192657 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:10:37.192619    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:10:51.682721 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:10:51.682643    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:11:10.983777 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:10.983742    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 15:11:10.984292 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:10.983952    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" podUID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" containerName="manager" containerID="cri-o://deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62" gracePeriod=10
Jul 17 15:11:11.443380 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.443359    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 15:11:11.534041 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.534010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvvmc\" (UniqueName: \"kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc\") pod \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") "
Jul 17 15:11:11.534216 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.534209    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume\") pod \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\" (UID: \"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6\") "
Jul 17 15:11:11.534552 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.534526    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" (UID: "85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:11:11.535927 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.535908    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc" (OuterVolumeSpecName: "kube-api-access-dvvmc") pod "85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" (UID: "85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6"). InnerVolumeSpecName "kube-api-access-dvvmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:11:11.635371 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.635342    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvvmc\" (UniqueName: \"kubernetes.io/projected/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-kube-api-access-dvvmc\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:11:11.635371 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.635367    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6-extensions-socket-volume\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:11:11.834633 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.834598    2564 generic.go:356] "Generic (PLEG): container finished" podID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" containerID="deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62" exitCode=0
Jul 17 15:11:11.834826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.834641    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"
Jul 17 15:11:11.834826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.834691    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" event={"ID":"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6","Type":"ContainerDied","Data":"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"}
Jul 17 15:11:11.834826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.834745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s" event={"ID":"85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6","Type":"ContainerDied","Data":"52215c7a2f72b106b0ade4b868fe4611e09c21c1212233bbcb9c407b94797324"}
Jul 17 15:11:11.834826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.834770    2564 scope.go:117] "RemoveContainer" containerID="deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"
Jul 17 15:11:11.844995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.844976    2564 scope.go:117] "RemoveContainer" containerID="deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"
Jul 17 15:11:11.845267 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:11:11.845245    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62\": container with ID starting with deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62 not found: ID does not exist" containerID="deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"
Jul 17 15:11:11.845362 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.845272    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62"} err="failed to get container status \"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62\": rpc error: code = NotFound desc = could not find container \"deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62\": container with ID starting with deaaf589134d5f7ddea017795850abbc06fb3a8d71ccd9c07f8d65a35b654f62 not found: ID does not exist"
Jul 17 15:11:11.861122 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.861100    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 15:11:11.867262 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.867242    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-b4g5s"]
Jul 17 15:11:11.951243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:11:11.951175    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" path="/var/lib/kubelet/pods/85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6/volumes"
Jul 17 15:12:12.169747 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.169657    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"]
Jul 17 15:12:12.170264 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170233    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" containerName="manager"
Jul 17 15:12:12.170264 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170252    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" containerName="manager"
Jul 17 15:12:12.170385 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170283    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" containerName="authorino"
Jul 17 15:12:12.170385 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170293    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" containerName="authorino"
Jul 17 15:12:12.170488 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170385    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c33948-ea3c-44b4-a0bb-5f0ea8a17aca" containerName="authorino"
Jul 17 15:12:12.170488 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.170400    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85e3d4cd-0636-4ab3-bf03-5c1ce0d0efe6" containerName="manager"
Jul 17 15:12:12.174649 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.174628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.178114 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.178094    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mwr6q\""
Jul 17 15:12:12.189149 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.189130    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"]
Jul 17 15:12:12.253722 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.253688    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x768w\" (UniqueName: \"kubernetes.io/projected/1db69919-4131-45a5-9b37-f00857b5fd3b-kube-api-access-x768w\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.253869 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.253796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1db69919-4131-45a5-9b37-f00857b5fd3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.354800 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.354764    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x768w\" (UniqueName: \"kubernetes.io/projected/1db69919-4131-45a5-9b37-f00857b5fd3b-kube-api-access-x768w\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.354976 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.354842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1db69919-4131-45a5-9b37-f00857b5fd3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.355270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.355250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1db69919-4131-45a5-9b37-f00857b5fd3b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.367978 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.367953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x768w\" (UniqueName: \"kubernetes.io/projected/1db69919-4131-45a5-9b37-f00857b5fd3b-kube-api-access-x768w\") pod \"kuadrant-operator-controller-manager-86878c769f-vp5w8\" (UID: \"1db69919-4131-45a5-9b37-f00857b5fd3b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.485770 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.485685    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:12.626828 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:12.626795    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"]
Jul 17 15:12:12.631336 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:12:12.631302    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db69919_4131_45a5_9b37_f00857b5fd3b.slice/crio-4a3c3b2e112d15e5581a7f18152f2bcd490828a4cfdbfeaa3e4149a6dbbf0704 WatchSource:0}: Error finding container 4a3c3b2e112d15e5581a7f18152f2bcd490828a4cfdbfeaa3e4149a6dbbf0704: Status 404 returned error can't find the container with id 4a3c3b2e112d15e5581a7f18152f2bcd490828a4cfdbfeaa3e4149a6dbbf0704
Jul 17 15:12:13.122391 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:13.122352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8" event={"ID":"1db69919-4131-45a5-9b37-f00857b5fd3b","Type":"ContainerStarted","Data":"4db395748582a3f8874f6126c02968154dbe58e4df0ad88144e0e29412b30a1a"}
Jul 17 15:12:13.122560 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:13.122407    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8" event={"ID":"1db69919-4131-45a5-9b37-f00857b5fd3b","Type":"ContainerStarted","Data":"4a3c3b2e112d15e5581a7f18152f2bcd490828a4cfdbfeaa3e4149a6dbbf0704"}
Jul 17 15:12:13.122560 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:13.122473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:24.131409 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:24.131373    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8"
Jul 17 15:12:24.156304 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:24.156257    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vp5w8" podStartSLOduration=12.156238869 podStartE2EDuration="12.156238869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:12:13.152774485 +0000 UTC m=+1343.719805616" watchObservedRunningTime="2026-07-17 15:12:24.156238869 +0000 UTC m=+1354.723270002"
Jul 17 15:12:36.132001 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:36.131964    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:12:47.987375 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:12:47.987342    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:00.580630 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:00.580590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:12.008848 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:12.008806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:23.187428 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:23.187380    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:34.589753 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:34.589714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:42.834050 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:42.833958    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:13:54.093260 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:13:54.093221    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:14:04.898320 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:04.898286    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:14:13.880028 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:13.879988    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:14:28.098939 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:28.098904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:14:49.458008 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:49.457970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:14:49.959282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:49.959257    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:14:49.959744 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:49.959728    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:14:49.964871 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:49.964842    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:14:49.965456 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:49.965438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:14:56.599308 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:14:56.599274    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:15:00.159965 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.159931    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"]
Jul 17 15:15:00.163174 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.163156    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"
Jul 17 15:15:00.165844 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.165826    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:15:00.173075 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.173043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"
Jul 17 15:15:00.174537 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.174515    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"]
Jul 17 15:15:00.315516 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.315491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"]
Jul 17 15:15:00.317523 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:15:00.317490    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf51fac8e_8cfd_4f54_9b18_9400b3458139.slice/crio-b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e WatchSource:0}: Error finding container b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e: Status 404 returned error can't find the container with id b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e
Jul 17 15:15:00.319479 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.319459    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:15:00.947947 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:00.947910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f" event={"ID":"f51fac8e-8cfd-4f54-9b18-9400b3458139","Type":"ContainerStarted","Data":"b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e"}
Jul 17 15:15:02.962309 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:02.962272    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f" event={"ID":"f51fac8e-8cfd-4f54-9b18-9400b3458139","Type":"ContainerStarted","Data":"8a2f5c85fe6e46afef10386a372404e09d02a02d4b8b62926dd656f3bdb38139"}
Jul 17 15:15:02.982356 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:02.982253    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f" podStartSLOduration=0.838640804 podStartE2EDuration="2.982237415s" totalImagesPullingTime="2.143596611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:15:02.980775289 +0000 UTC m=+1513.547806435" watchObservedRunningTime="2026-07-17 15:15:02.982237415 +0000 UTC m=+1513.549268546"
Jul 17 15:15:03.968019 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:03.967980    2564 generic.go:356] "Generic (PLEG): container finished" podID="f51fac8e-8cfd-4f54-9b18-9400b3458139" containerID="8a2f5c85fe6e46afef10386a372404e09d02a02d4b8b62926dd656f3bdb38139" exitCode=0
Jul 17 15:15:03.968475 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:03.968073    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f" event={"ID":"f51fac8e-8cfd-4f54-9b18-9400b3458139","Type":"ContainerDied","Data":"8a2f5c85fe6e46afef10386a372404e09d02a02d4b8b62926dd656f3bdb38139"}
Jul 17 15:15:05.108864 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:05.108836    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"
Jul 17 15:15:05.978969 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:05.978939    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f"
Jul 17 15:15:05.979136 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:05.978974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738355-75f5f" event={"ID":"f51fac8e-8cfd-4f54-9b18-9400b3458139","Type":"ContainerDied","Data":"b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e"}
Jul 17 15:15:05.979136 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:05.979018    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b44213954afa515d00a4d6d63c1e108f5543589d7337b6c101113e8c0a05b92e"
Jul 17 15:15:06.788264 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:06.788235    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:15:16.191212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:16.191129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:15:23.273039 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:23.273003    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:15:40.591110 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:40.591059    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:15:51.686861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:15:51.686824    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:01.184007 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:01.183969    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:11.183399 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:11.183358    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:20.583990 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:20.583955    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:30.282372 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:30.282341    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:43.987265 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:43.987185    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:16:58.688088 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:16:58.688030    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:17:08.687121 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:17:08.687082    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:17:15.197094 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:17:15.197044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:17:28.107407 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:17:28.107373    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:17:39.288290 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:17:39.288255    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:18:17.684320 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:18:17.684234    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:18:33.801683 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:18:33.801649    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:18:42.785767 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:18:42.785730    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:19:01.448122 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.448081    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:01.448514 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.448455    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f51fac8e-8cfd-4f54-9b18-9400b3458139" containerName="cleanup"
Jul 17 15:19:01.448514 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.448466    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f51fac8e-8cfd-4f54-9b18-9400b3458139" containerName="cleanup"
Jul 17 15:19:01.448586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.448534    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f51fac8e-8cfd-4f54-9b18-9400b3458139" containerName="cleanup"
Jul 17 15:19:01.451698 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.451683    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:01.454961 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.454943    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:19:01.455181 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.455168    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:19:01.460647 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.460624    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:01.545946 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.545915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw\" (UID: \"277abae8-6364-4691-855c-3d6ee5c6fd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:01.647133 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.647091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw\" (UID: \"277abae8-6364-4691-855c-3d6ee5c6fd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:01.647752 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.647730    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw\" (UID: \"277abae8-6364-4691-855c-3d6ee5c6fd06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:01.762168 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.762057    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:01.901551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:01.901523    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:01.903961 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:19:01.903931    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277abae8_6364_4691_855c_3d6ee5c6fd06.slice/crio-3e9bcc5d749dc8c55f98aa1656bbc5a83b828d0531204d38153e5e6696d8e067 WatchSource:0}: Error finding container 3e9bcc5d749dc8c55f98aa1656bbc5a83b828d0531204d38153e5e6696d8e067: Status 404 returned error can't find the container with id 3e9bcc5d749dc8c55f98aa1656bbc5a83b828d0531204d38153e5e6696d8e067
Jul 17 15:19:02.091398 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:02.091371    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:19:02.158306 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:02.158277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerStarted","Data":"00291d4995f468f6f88214eadbd542df5552d484afc501f5611cf55a23407525"}
Jul 17 15:19:02.158461 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:02.158325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerStarted","Data":"3e9bcc5d749dc8c55f98aa1656bbc5a83b828d0531204d38153e5e6696d8e067"}
Jul 17 15:19:02.179409 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:02.179355    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" podStartSLOduration=1.179336421 podStartE2EDuration="1.179336421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:19:02.177734088 +0000 UTC m=+1752.744765226" watchObservedRunningTime="2026-07-17 15:19:02.179336421 +0000 UTC m=+1752.746367554"
Jul 17 15:19:03.164666 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:03.164623    2564 generic.go:356] "Generic (PLEG): container finished" podID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerID="00291d4995f468f6f88214eadbd542df5552d484afc501f5611cf55a23407525" exitCode=7
Jul 17 15:19:03.165120 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:03.164689    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerDied","Data":"00291d4995f468f6f88214eadbd542df5552d484afc501f5611cf55a23407525"}
Jul 17 15:19:03.165120 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:03.164952    2564 scope.go:117] "RemoveContainer" containerID="00291d4995f468f6f88214eadbd542df5552d484afc501f5611cf55a23407525"
Jul 17 15:19:04.171652 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:04.171562    2564 generic.go:356] "Generic (PLEG): container finished" podID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerID="18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5" exitCode=7
Jul 17 15:19:04.172058 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:04.171648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerDied","Data":"18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5"}
Jul 17 15:19:04.172058 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:04.171714    2564 scope.go:117] "RemoveContainer" containerID="00291d4995f468f6f88214eadbd542df5552d484afc501f5611cf55a23407525"
Jul 17 15:19:04.172058 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:04.171821    2564 scope.go:117] "RemoveContainer" containerID="18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5"
Jul 17 15:19:04.172058 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:19:04.172009    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw_odh-ai-gateway-infra(277abae8-6364-4691-855c-3d6ee5c6fd06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06"
Jul 17 15:19:05.178910 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:05.178880    2564 scope.go:117] "RemoveContainer" containerID="18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5"
Jul 17 15:19:05.179500 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:19:05.179144    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw_odh-ai-gateway-infra(277abae8-6364-4691-855c-3d6ee5c6fd06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06"
Jul 17 15:19:18.944996 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:18.944965    2564 scope.go:117] "RemoveContainer" containerID="18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5"
Jul 17 15:19:19.251811 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:19.251732    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerStarted","Data":"0aed0345db074501e39d93900c12d518f0daec6efa2c107a9f72c8aa2f4b29cd"}
Jul 17 15:19:20.259983 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:20.259949    2564 generic.go:356] "Generic (PLEG): container finished" podID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerID="0aed0345db074501e39d93900c12d518f0daec6efa2c107a9f72c8aa2f4b29cd" exitCode=7
Jul 17 15:19:20.260470 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:20.260029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerDied","Data":"0aed0345db074501e39d93900c12d518f0daec6efa2c107a9f72c8aa2f4b29cd"}
Jul 17 15:19:20.260470 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:20.260121    2564 scope.go:117] "RemoveContainer" containerID="18f0bd8b98bfb1498dfb5920011acd90a4dbb16643f40d36203cc709f312b6c5"
Jul 17 15:19:20.260470 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:20.260426    2564 scope.go:117] "RemoveContainer" containerID="0aed0345db074501e39d93900c12d518f0daec6efa2c107a9f72c8aa2f4b29cd"
Jul 17 15:19:20.260666 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:19:20.260646    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw_odh-ai-gateway-infra(277abae8-6364-4691-855c-3d6ee5c6fd06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06"
Jul 17 15:19:20.286342 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:20.286318    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:21.395236 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:21.395195    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:21.530241 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:21.530166    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca\") pod \"277abae8-6364-4691-855c-3d6ee5c6fd06\" (UID: \"277abae8-6364-4691-855c-3d6ee5c6fd06\") "
Jul 17 15:19:21.530495 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:21.530471    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "277abae8-6364-4691-855c-3d6ee5c6fd06" (UID: "277abae8-6364-4691-855c-3d6ee5c6fd06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:19:21.630980 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:21.630948    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277abae8-6364-4691-855c-3d6ee5c6fd06-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:19:22.272671 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:22.272584    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"
Jul 17 15:19:22.272671 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:22.272618    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw" event={"ID":"277abae8-6364-4691-855c-3d6ee5c6fd06","Type":"ContainerDied","Data":"3e9bcc5d749dc8c55f98aa1656bbc5a83b828d0531204d38153e5e6696d8e067"}
Jul 17 15:19:22.272894 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:22.272687    2564 scope.go:117] "RemoveContainer" containerID="0aed0345db074501e39d93900c12d518f0daec6efa2c107a9f72c8aa2f4b29cd"
Jul 17 15:19:22.291485 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:22.291459    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:22.299095 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:22.299073    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-q8psw"]
Jul 17 15:19:23.951382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:23.951346    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" path="/var/lib/kubelet/pods/277abae8-6364-4691-855c-3d6ee5c6fd06/volumes"
Jul 17 15:19:49.991252 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:49.991221    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:19:49.994116 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:49.994093    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:19:49.996588 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:49.996565    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:19:49.999437 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:19:49.999422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:20:01.523646 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.523614    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:01.524032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.523969    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.524032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.523979    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.524032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.523988    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.524032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.524001    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.524207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.524085    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.524207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.524097    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:01.527160 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.527144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:01.530154 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.530126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:20:01.530258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.530188    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:20:01.536272 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.536252    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:01.657630 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.657592    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5\" (UID: \"3d4a625e-cd00-4b1b-ba64-733ee1d387f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:01.758616 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.758579    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5\" (UID: \"3d4a625e-cd00-4b1b-ba64-733ee1d387f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:01.759290 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.759269    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5\" (UID: \"3d4a625e-cd00-4b1b-ba64-733ee1d387f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:01.838777 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:01.838741    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:02.183419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:02.183393    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:02.186156 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:20:02.186129    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4a625e_cd00_4b1b_ba64_733ee1d387f3.slice/crio-9db9f17b985d27fa81bb93f4ff595f9299b2df490d7e418d54eb6fa7bcb10bf1 WatchSource:0}: Error finding container 9db9f17b985d27fa81bb93f4ff595f9299b2df490d7e418d54eb6fa7bcb10bf1: Status 404 returned error can't find the container with id 9db9f17b985d27fa81bb93f4ff595f9299b2df490d7e418d54eb6fa7bcb10bf1
Jul 17 15:20:02.461131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:02.461094    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" event={"ID":"3d4a625e-cd00-4b1b-ba64-733ee1d387f3","Type":"ContainerStarted","Data":"9db9f17b985d27fa81bb93f4ff595f9299b2df490d7e418d54eb6fa7bcb10bf1"}
Jul 17 15:20:03.467420 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:03.467386    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerID="4d145633b79f2a7eadda728429f7ece76665dd14239327886a37e3b1f4c6f029" exitCode=7
Jul 17 15:20:03.467822 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:03.467466    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" event={"ID":"3d4a625e-cd00-4b1b-ba64-733ee1d387f3","Type":"ContainerDied","Data":"4d145633b79f2a7eadda728429f7ece76665dd14239327886a37e3b1f4c6f029"}
Jul 17 15:20:03.467822 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:03.467637    2564 scope.go:117] "RemoveContainer" containerID="4d145633b79f2a7eadda728429f7ece76665dd14239327886a37e3b1f4c6f029"
Jul 17 15:20:04.474847 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:04.474808    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerID="01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092" exitCode=7
Jul 17 15:20:04.475282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:04.474891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" event={"ID":"3d4a625e-cd00-4b1b-ba64-733ee1d387f3","Type":"ContainerDied","Data":"01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092"}
Jul 17 15:20:04.475282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:04.474963    2564 scope.go:117] "RemoveContainer" containerID="4d145633b79f2a7eadda728429f7ece76665dd14239327886a37e3b1f4c6f029"
Jul 17 15:20:04.475282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:04.475122    2564 scope.go:117] "RemoveContainer" containerID="01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092"
Jul 17 15:20:04.475414 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:20:04.475342    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5_odh-ai-gateway-infra(3d4a625e-cd00-4b1b-ba64-733ee1d387f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3"
Jul 17 15:20:05.488540 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:05.488512    2564 scope.go:117] "RemoveContainer" containerID="01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092"
Jul 17 15:20:05.488906 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:20:05.488681    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5_odh-ai-gateway-infra(3d4a625e-cd00-4b1b-ba64-733ee1d387f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3"
Jul 17 15:20:16.944344 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:16.944314    2564 scope.go:117] "RemoveContainer" containerID="01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092"
Jul 17 15:20:17.548816 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:17.548788    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerID="7dfec5a78f4ad42df4e39a012b951620ccd2940f196411721c0d4617f82f49af" exitCode=7
Jul 17 15:20:17.548961 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:17.548869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" event={"ID":"3d4a625e-cd00-4b1b-ba64-733ee1d387f3","Type":"ContainerDied","Data":"7dfec5a78f4ad42df4e39a012b951620ccd2940f196411721c0d4617f82f49af"}
Jul 17 15:20:17.548961 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:17.548932    2564 scope.go:117] "RemoveContainer" containerID="01eddf2ca682e281989559d8766b55aff524a71e75b2137c88c090d172011092"
Jul 17 15:20:17.549240 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:17.549223    2564 scope.go:117] "RemoveContainer" containerID="7dfec5a78f4ad42df4e39a012b951620ccd2940f196411721c0d4617f82f49af"
Jul 17 15:20:17.549468 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:20:17.549447    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5_odh-ai-gateway-infra(3d4a625e-cd00-4b1b-ba64-733ee1d387f3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3"
Jul 17 15:20:18.594518 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:18.594482    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:18.731202 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:18.731179    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:18.809677 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:18.809649    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca\") pod \"3d4a625e-cd00-4b1b-ba64-733ee1d387f3\" (UID: \"3d4a625e-cd00-4b1b-ba64-733ee1d387f3\") "
Jul 17 15:20:18.810004 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:18.809983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d4a625e-cd00-4b1b-ba64-733ee1d387f3" (UID: "3d4a625e-cd00-4b1b-ba64-733ee1d387f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:20:18.910507 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:18.910436    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d4a625e-cd00-4b1b-ba64-733ee1d387f3-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:20:19.563075 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.563044    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"
Jul 17 15:20:19.563258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.563103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5" event={"ID":"3d4a625e-cd00-4b1b-ba64-733ee1d387f3","Type":"ContainerDied","Data":"9db9f17b985d27fa81bb93f4ff595f9299b2df490d7e418d54eb6fa7bcb10bf1"}
Jul 17 15:20:19.563258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.563198    2564 scope.go:117] "RemoveContainer" containerID="7dfec5a78f4ad42df4e39a012b951620ccd2940f196411721c0d4617f82f49af"
Jul 17 15:20:19.588486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.588451    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:19.591258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.591234    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-fzsp5"]
Jul 17 15:20:19.950478 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:19.950398    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" path="/var/lib/kubelet/pods/3d4a625e-cd00-4b1b-ba64-733ee1d387f3/volumes"
Jul 17 15:20:51.584524 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584486    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584861    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584874    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584893    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584898    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584907    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.584952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584912    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.585193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.584991    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="277abae8-6364-4691-855c-3d6ee5c6fd06" containerName="revoke-keys"
Jul 17 15:20:51.585193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.585001    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.585193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.585008    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.585193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.585015    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:20:51.588050 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.588033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:20:51.590467 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.590443    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:20:51.590605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.590505    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:20:51.596811 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.596787    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:20:51.688375 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.688343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw\" (UID: \"96ef4e15-545f-4ae5-9f8c-839c2fad2720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:20:51.788818 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.788786    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw\" (UID: \"96ef4e15-545f-4ae5-9f8c-839c2fad2720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:20:51.789442 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.789421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw\" (UID: \"96ef4e15-545f-4ae5-9f8c-839c2fad2720\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:20:51.900555 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:51.900474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:20:52.039332 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:52.039303    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:20:52.042372 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:20:52.042345    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ef4e15_545f_4ae5_9f8c_839c2fad2720.slice/crio-7513c16709326980c0c43373d218024d62b8512c85a6a4d8ef56967bc6ece0f1 WatchSource:0}: Error finding container 7513c16709326980c0c43373d218024d62b8512c85a6a4d8ef56967bc6ece0f1: Status 404 returned error can't find the container with id 7513c16709326980c0c43373d218024d62b8512c85a6a4d8ef56967bc6ece0f1
Jul 17 15:20:52.723002 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:52.722969    2564 generic.go:356] "Generic (PLEG): container finished" podID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerID="257eddd36effd831a4d6114758edd0d9d23061be3c0fe7f4616bb304a3a04ba8" exitCode=7
Jul 17 15:20:52.723372 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:52.723017    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" event={"ID":"96ef4e15-545f-4ae5-9f8c-839c2fad2720","Type":"ContainerDied","Data":"257eddd36effd831a4d6114758edd0d9d23061be3c0fe7f4616bb304a3a04ba8"}
Jul 17 15:20:52.723372 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:52.723054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" event={"ID":"96ef4e15-545f-4ae5-9f8c-839c2fad2720","Type":"ContainerStarted","Data":"7513c16709326980c0c43373d218024d62b8512c85a6a4d8ef56967bc6ece0f1"}
Jul 17 15:20:52.723372 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:52.723230    2564 scope.go:117] "RemoveContainer" containerID="257eddd36effd831a4d6114758edd0d9d23061be3c0fe7f4616bb304a3a04ba8"
Jul 17 15:20:53.729715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:53.729621    2564 generic.go:356] "Generic (PLEG): container finished" podID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerID="b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3" exitCode=7
Jul 17 15:20:53.729715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:53.729699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" event={"ID":"96ef4e15-545f-4ae5-9f8c-839c2fad2720","Type":"ContainerDied","Data":"b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3"}
Jul 17 15:20:53.730217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:53.729757    2564 scope.go:117] "RemoveContainer" containerID="257eddd36effd831a4d6114758edd0d9d23061be3c0fe7f4616bb304a3a04ba8"
Jul 17 15:20:53.730217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:53.729941    2564 scope.go:117] "RemoveContainer" containerID="b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3"
Jul 17 15:20:53.730217 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:20:53.730160    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw_odh-ai-gateway-infra(96ef4e15-545f-4ae5-9f8c-839c2fad2720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720"
Jul 17 15:20:54.735768 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:20:54.735743    2564 scope.go:117] "RemoveContainer" containerID="b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3"
Jul 17 15:20:54.736185 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:20:54.735907    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw_odh-ai-gateway-infra(96ef4e15-545f-4ae5-9f8c-839c2fad2720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720"
Jul 17 15:21:05.944351 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:05.944318    2564 scope.go:117] "RemoveContainer" containerID="b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3"
Jul 17 15:21:06.795206 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:06.795111    2564 generic.go:356] "Generic (PLEG): container finished" podID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerID="0b9932ba519e423759278cfab522dca1b2291e61bc20e42b6bffc1f616224ae6" exitCode=7
Jul 17 15:21:06.795206 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:06.795180    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" event={"ID":"96ef4e15-545f-4ae5-9f8c-839c2fad2720","Type":"ContainerDied","Data":"0b9932ba519e423759278cfab522dca1b2291e61bc20e42b6bffc1f616224ae6"}
Jul 17 15:21:06.795715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:06.795243    2564 scope.go:117] "RemoveContainer" containerID="b4a0c51bd9d823b043b147a9be906699a47e1f20dd8ab62b2d52501898d8b5f3"
Jul 17 15:21:06.795715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:06.795542    2564 scope.go:117] "RemoveContainer" containerID="0b9932ba519e423759278cfab522dca1b2291e61bc20e42b6bffc1f616224ae6"
Jul 17 15:21:06.795820 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:21:06.795804    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw_odh-ai-gateway-infra(96ef4e15-545f-4ae5-9f8c-839c2fad2720)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720"
Jul 17 15:21:07.837018 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:07.836983    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:21:07.975950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:07.975925    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:21:08.134242 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.134165    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca\") pod \"96ef4e15-545f-4ae5-9f8c-839c2fad2720\" (UID: \"96ef4e15-545f-4ae5-9f8c-839c2fad2720\") "
Jul 17 15:21:08.134547 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.134523    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96ef4e15-545f-4ae5-9f8c-839c2fad2720" (UID: "96ef4e15-545f-4ae5-9f8c-839c2fad2720"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:21:08.235737 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.235699    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ef4e15-545f-4ae5-9f8c-839c2fad2720-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:21:08.806989 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.806960    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"
Jul 17 15:21:08.807177 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.806955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw" event={"ID":"96ef4e15-545f-4ae5-9f8c-839c2fad2720","Type":"ContainerDied","Data":"7513c16709326980c0c43373d218024d62b8512c85a6a4d8ef56967bc6ece0f1"}
Jul 17 15:21:08.807177 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.807113    2564 scope.go:117] "RemoveContainer" containerID="0b9932ba519e423759278cfab522dca1b2291e61bc20e42b6bffc1f616224ae6"
Jul 17 15:21:08.830021 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.829992    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:21:08.837089 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:08.837054    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8bd9392b-mldxw"]
Jul 17 15:21:09.950367 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:09.950340    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" path="/var/lib/kubelet/pods/96ef4e15-545f-4ae5-9f8c-839c2fad2720/volumes"
Jul 17 15:21:14.892000 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:14.891922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:21:33.247954 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.247920    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248306    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248318    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d4a625e-cd00-4b1b-ba64-733ee1d387f3" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248336    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248342    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248349    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248354    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248361    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248366    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248670 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248424    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248670 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248431    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.248670 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.248439    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef4e15-545f-4ae5-9f8c-839c2fad2720" containerName="revoke-keys"
Jul 17 15:21:33.251012 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.250996    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:33.253451 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.253431    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:21:33.253559 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.253431    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:21:33.259491 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.259459    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:33.344414 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.344371    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc\" (UID: \"01ea9200-b1e1-4f67-8732-21926c856a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:33.445289 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.445254    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc\" (UID: \"01ea9200-b1e1-4f67-8732-21926c856a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:33.445858 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.445841    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc\" (UID: \"01ea9200-b1e1-4f67-8732-21926c856a0d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:33.562223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.562197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:33.697927 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.697904    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:33.700081 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:21:33.700026    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ea9200_b1e1_4f67_8732_21926c856a0d.slice/crio-f4db6dff8100516fb8d94e6e4ce22fc535e0ddcc4b5e6244dbdd8780b1980fea WatchSource:0}: Error finding container f4db6dff8100516fb8d94e6e4ce22fc535e0ddcc4b5e6244dbdd8780b1980fea: Status 404 returned error can't find the container with id f4db6dff8100516fb8d94e6e4ce22fc535e0ddcc4b5e6244dbdd8780b1980fea
Jul 17 15:21:33.925175 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:33.925100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" event={"ID":"01ea9200-b1e1-4f67-8732-21926c856a0d","Type":"ContainerStarted","Data":"f4db6dff8100516fb8d94e6e4ce22fc535e0ddcc4b5e6244dbdd8780b1980fea"}
Jul 17 15:21:34.930944 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:34.930902    2564 generic.go:356] "Generic (PLEG): container finished" podID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerID="abb1109a146539bd26092e07952f63ebb64eb8a6db0bc4a720ba115c3088c9cd" exitCode=7
Jul 17 15:21:34.931366 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:34.930991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" event={"ID":"01ea9200-b1e1-4f67-8732-21926c856a0d","Type":"ContainerDied","Data":"abb1109a146539bd26092e07952f63ebb64eb8a6db0bc4a720ba115c3088c9cd"}
Jul 17 15:21:34.931366 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:34.931151    2564 scope.go:117] "RemoveContainer" containerID="abb1109a146539bd26092e07952f63ebb64eb8a6db0bc4a720ba115c3088c9cd"
Jul 17 15:21:35.937949 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:35.937914    2564 generic.go:356] "Generic (PLEG): container finished" podID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerID="9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1" exitCode=7
Jul 17 15:21:35.938378 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:35.938002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" event={"ID":"01ea9200-b1e1-4f67-8732-21926c856a0d","Type":"ContainerDied","Data":"9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1"}
Jul 17 15:21:35.938378 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:35.938087    2564 scope.go:117] "RemoveContainer" containerID="abb1109a146539bd26092e07952f63ebb64eb8a6db0bc4a720ba115c3088c9cd"
Jul 17 15:21:35.938378 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:35.938284    2564 scope.go:117] "RemoveContainer" containerID="9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1"
Jul 17 15:21:35.938524 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:21:35.938500    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc_odh-ai-gateway-infra(01ea9200-b1e1-4f67-8732-21926c856a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d"
Jul 17 15:21:35.990704 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:35.990672    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:21:36.944837 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:36.944810    2564 scope.go:117] "RemoveContainer" containerID="9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1"
Jul 17 15:21:36.945338 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:21:36.944980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc_odh-ai-gateway-infra(01ea9200-b1e1-4f67-8732-21926c856a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d"
Jul 17 15:21:50.944798 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:50.944768    2564 scope.go:117] "RemoveContainer" containerID="9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1"
Jul 17 15:21:52.025370 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:52.025333    2564 generic.go:356] "Generic (PLEG): container finished" podID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerID="db3022cdbe9399def292f84dd3c27b0ace438a2205117f90d344c56cfb873606" exitCode=7
Jul 17 15:21:52.025841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:52.025405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" event={"ID":"01ea9200-b1e1-4f67-8732-21926c856a0d","Type":"ContainerDied","Data":"db3022cdbe9399def292f84dd3c27b0ace438a2205117f90d344c56cfb873606"}
Jul 17 15:21:52.025841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:52.025463    2564 scope.go:117] "RemoveContainer" containerID="9d6b42b29b34a96ba0537dfc5f31b5d0129797925f99951a915e80da99da04e1"
Jul 17 15:21:52.025841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:52.025751    2564 scope.go:117] "RemoveContainer" containerID="db3022cdbe9399def292f84dd3c27b0ace438a2205117f90d344c56cfb873606"
Jul 17 15:21:52.026027 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:21:52.026004    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc_odh-ai-gateway-infra(01ea9200-b1e1-4f67-8732-21926c856a0d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d"
Jul 17 15:21:53.055194 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:53.055162    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:53.184362 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:53.184337    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:53.319718 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:53.319691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca\") pod \"01ea9200-b1e1-4f67-8732-21926c856a0d\" (UID: \"01ea9200-b1e1-4f67-8732-21926c856a0d\") "
Jul 17 15:21:53.320051 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:53.320029    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01ea9200-b1e1-4f67-8732-21926c856a0d" (UID: "01ea9200-b1e1-4f67-8732-21926c856a0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:21:53.421045 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:53.421017    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01ea9200-b1e1-4f67-8732-21926c856a0d-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:21:54.037221 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:54.037193    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"
Jul 17 15:21:54.037370 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:54.037218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc" event={"ID":"01ea9200-b1e1-4f67-8732-21926c856a0d","Type":"ContainerDied","Data":"f4db6dff8100516fb8d94e6e4ce22fc535e0ddcc4b5e6244dbdd8780b1980fea"}
Jul 17 15:21:54.037370 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:54.037271    2564 scope.go:117] "RemoveContainer" containerID="db3022cdbe9399def292f84dd3c27b0ace438a2205117f90d344c56cfb873606"
Jul 17 15:21:54.057450 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:54.057425    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:54.059126 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:54.059105    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-gcvlc"]
Jul 17 15:21:55.950517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:21:55.950484    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" path="/var/lib/kubelet/pods/01ea9200-b1e1-4f67-8732-21926c856a0d/volumes"
Jul 17 15:22:33.360714 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.360672    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361044    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361057    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361124    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361135    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361197    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361207    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.361210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.361214    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:33.364427 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.364409    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:33.367921 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.367900    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:22:33.368347 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.368330    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:22:33.396192 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.396158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:33.461972 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.461942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r\" (UID: \"1053404b-1993-4836-82f7-d54fa6a8cb4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:33.563033 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.562993    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r\" (UID: \"1053404b-1993-4836-82f7-d54fa6a8cb4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:33.563650 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.563623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r\" (UID: \"1053404b-1993-4836-82f7-d54fa6a8cb4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:33.675379 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.675296    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:33.817043 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:33.817012    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:33.818865 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:22:33.818827    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1053404b_1993_4836_82f7_d54fa6a8cb4e.slice/crio-757944791f6fec1faf84cf53f8b445afd9bf31c704827282a6815aec4e2e034f WatchSource:0}: Error finding container 757944791f6fec1faf84cf53f8b445afd9bf31c704827282a6815aec4e2e034f: Status 404 returned error can't find the container with id 757944791f6fec1faf84cf53f8b445afd9bf31c704827282a6815aec4e2e034f
Jul 17 15:22:34.232695 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:34.232608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerStarted","Data":"1034d93700f33a1c16e54ae359171ef0e07e2bf868cdc90724528bc036fdff5e"}
Jul 17 15:22:34.232695 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:34.232669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerStarted","Data":"757944791f6fec1faf84cf53f8b445afd9bf31c704827282a6815aec4e2e034f"}
Jul 17 15:22:34.251966 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:34.251838    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" podStartSLOduration=1.25182107 podStartE2EDuration="1.25182107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:22:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:22:34.251241684 +0000 UTC m=+1964.818272835" watchObservedRunningTime="2026-07-17 15:22:34.25182107 +0000 UTC m=+1964.818852203"
Jul 17 15:22:35.239823 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:35.239786    2564 generic.go:356] "Generic (PLEG): container finished" podID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerID="1034d93700f33a1c16e54ae359171ef0e07e2bf868cdc90724528bc036fdff5e" exitCode=7
Jul 17 15:22:35.240278 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:35.239870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerDied","Data":"1034d93700f33a1c16e54ae359171ef0e07e2bf868cdc90724528bc036fdff5e"}
Jul 17 15:22:35.240278 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:35.240028    2564 scope.go:117] "RemoveContainer" containerID="1034d93700f33a1c16e54ae359171ef0e07e2bf868cdc90724528bc036fdff5e"
Jul 17 15:22:36.246861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:36.246772    2564 generic.go:356] "Generic (PLEG): container finished" podID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerID="9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb" exitCode=7
Jul 17 15:22:36.246861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:36.246845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerDied","Data":"9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb"}
Jul 17 15:22:36.247389 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:36.246898    2564 scope.go:117] "RemoveContainer" containerID="1034d93700f33a1c16e54ae359171ef0e07e2bf868cdc90724528bc036fdff5e"
Jul 17 15:22:36.247389 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:36.247138    2564 scope.go:117] "RemoveContainer" containerID="9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb"
Jul 17 15:22:36.247389 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:22:36.247380    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r_odh-ai-gateway-infra(1053404b-1993-4836-82f7-d54fa6a8cb4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e"
Jul 17 15:22:36.647978 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:36.647940    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:22:37.253149 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:37.253124    2564 scope.go:117] "RemoveContainer" containerID="9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb"
Jul 17 15:22:37.253498 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:22:37.253321    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r_odh-ai-gateway-infra(1053404b-1993-4836-82f7-d54fa6a8cb4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e"
Jul 17 15:22:51.944258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:51.944222    2564 scope.go:117] "RemoveContainer" containerID="9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb"
Jul 17 15:22:52.326501 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:52.326467    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerStarted","Data":"a686e2091e073bc4fc7a1c41965ef2257b46d5da032865a340a11284afdc539a"}
Jul 17 15:22:53.333346 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:53.333314    2564 generic.go:356] "Generic (PLEG): container finished" podID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerID="a686e2091e073bc4fc7a1c41965ef2257b46d5da032865a340a11284afdc539a" exitCode=6
Jul 17 15:22:53.333828 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:53.333385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerDied","Data":"a686e2091e073bc4fc7a1c41965ef2257b46d5da032865a340a11284afdc539a"}
Jul 17 15:22:53.333828 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:53.333445    2564 scope.go:117] "RemoveContainer" containerID="9a23cd803e7311a8fd33a0f1a17392d289f84367ca4c55a424e00843c4a346bb"
Jul 17 15:22:53.333828 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:53.333754    2564 scope.go:117] "RemoveContainer" containerID="a686e2091e073bc4fc7a1c41965ef2257b46d5da032865a340a11284afdc539a"
Jul 17 15:22:53.334002 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:22:53.333980    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r_odh-ai-gateway-infra(1053404b-1993-4836-82f7-d54fa6a8cb4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e"
Jul 17 15:22:53.379479 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:53.379451    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:54.474186 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:54.474157    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:54.643486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:54.643408    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca\") pod \"1053404b-1993-4836-82f7-d54fa6a8cb4e\" (UID: \"1053404b-1993-4836-82f7-d54fa6a8cb4e\") "
Jul 17 15:22:54.643776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:54.643748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1053404b-1993-4836-82f7-d54fa6a8cb4e" (UID: "1053404b-1993-4836-82f7-d54fa6a8cb4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:22:54.744723 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:54.744696    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1053404b-1993-4836-82f7-d54fa6a8cb4e-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:22:55.346846 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.346814    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"
Jul 17 15:22:55.346846 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.346825    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r" event={"ID":"1053404b-1993-4836-82f7-d54fa6a8cb4e","Type":"ContainerDied","Data":"757944791f6fec1faf84cf53f8b445afd9bf31c704827282a6815aec4e2e034f"}
Jul 17 15:22:55.347125 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.346895    2564 scope.go:117] "RemoveContainer" containerID="a686e2091e073bc4fc7a1c41965ef2257b46d5da032865a340a11284afdc539a"
Jul 17 15:22:55.384391 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.384352    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:55.387966 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.387944    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-hhz6r"]
Jul 17 15:22:55.758428 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758350    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:22:55.758776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758727    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.758776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758740    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.758776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758763    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:55.758776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758768    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="01ea9200-b1e1-4f67-8732-21926c856a0d" containerName="revoke-keys"
Jul 17 15:22:55.758909 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758780    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.758909 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758786    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.758909 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758841    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.758909 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.758849    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:22:55.762211 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.762193    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:22:55.765167 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.765146    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:22:55.765293 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.765167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:22:55.781586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.781556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:22:55.854552 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.854523    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx\" (UID: \"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:22:55.950264 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.950233    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" path="/var/lib/kubelet/pods/1053404b-1993-4836-82f7-d54fa6a8cb4e/volumes"
Jul 17 15:22:55.955604 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.955582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx\" (UID: \"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:22:55.956230 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:55.956206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx\" (UID: \"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:22:56.072513 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:56.072489    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:22:56.212911 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:56.212860    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:22:56.215152 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:22:56.215125    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d309b0_2c9c_4496_8cbc_d4b9fbb2eebb.slice/crio-f2607cd12072b6057a5f1639f9bce263013ebc14898d4d07166dd6bf8db3ffb1 WatchSource:0}: Error finding container f2607cd12072b6057a5f1639f9bce263013ebc14898d4d07166dd6bf8db3ffb1: Status 404 returned error can't find the container with id f2607cd12072b6057a5f1639f9bce263013ebc14898d4d07166dd6bf8db3ffb1
Jul 17 15:22:56.353594 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:56.353515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerStarted","Data":"f2607cd12072b6057a5f1639f9bce263013ebc14898d4d07166dd6bf8db3ffb1"}
Jul 17 15:22:57.360246 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:57.360210    2564 generic.go:356] "Generic (PLEG): container finished" podID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerID="b18928dd8f0afc83ca739b6070c21ea1a98607f2c224831ec244ccb027e3157b" exitCode=7
Jul 17 15:22:57.360688 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:57.360291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerDied","Data":"b18928dd8f0afc83ca739b6070c21ea1a98607f2c224831ec244ccb027e3157b"}
Jul 17 15:22:57.360688 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:57.360485    2564 scope.go:117] "RemoveContainer" containerID="b18928dd8f0afc83ca739b6070c21ea1a98607f2c224831ec244ccb027e3157b"
Jul 17 15:22:58.367042 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:58.367008    2564 generic.go:356] "Generic (PLEG): container finished" podID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerID="3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c" exitCode=7
Jul 17 15:22:58.367530 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:58.367090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerDied","Data":"3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c"}
Jul 17 15:22:58.367530 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:58.367138    2564 scope.go:117] "RemoveContainer" containerID="b18928dd8f0afc83ca739b6070c21ea1a98607f2c224831ec244ccb027e3157b"
Jul 17 15:22:58.367530 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:58.367376    2564 scope.go:117] "RemoveContainer" containerID="3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c"
Jul 17 15:22:58.367691 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:22:58.367590    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx_odh-ai-gateway-infra(a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb"
Jul 17 15:22:59.374152 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:59.374128    2564 scope.go:117] "RemoveContainer" containerID="3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c"
Jul 17 15:22:59.374555 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:22:59.374300    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx_odh-ai-gateway-infra(a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb"
Jul 17 15:22:59.795294 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:22:59.795218    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:23:10.944754 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:10.944723    2564 scope.go:117] "RemoveContainer" containerID="3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c"
Jul 17 15:23:11.442221 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:11.442171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerStarted","Data":"4317208dd31c5130c9d70a82653635aa9afc6e8268781b6bbe777fc37e773850"}
Jul 17 15:23:11.467975 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:11.467927    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" podStartSLOduration=16.467911022 podStartE2EDuration="16.467911022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:23:11.467441166 +0000 UTC m=+2002.034472300" watchObservedRunningTime="2026-07-17 15:23:11.467911022 +0000 UTC m=+2002.034942153"
Jul 17 15:23:12.448771 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:12.448737    2564 generic.go:356] "Generic (PLEG): container finished" podID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerID="4317208dd31c5130c9d70a82653635aa9afc6e8268781b6bbe777fc37e773850" exitCode=7
Jul 17 15:23:12.449228 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:12.448811    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerDied","Data":"4317208dd31c5130c9d70a82653635aa9afc6e8268781b6bbe777fc37e773850"}
Jul 17 15:23:12.449228 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:12.448879    2564 scope.go:117] "RemoveContainer" containerID="3cdec45112aca0ed5ba7217783a8ba5850cdfe568cad5aae076200b4321dec0c"
Jul 17 15:23:12.449331 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:12.449254    2564 scope.go:117] "RemoveContainer" containerID="4317208dd31c5130c9d70a82653635aa9afc6e8268781b6bbe777fc37e773850"
Jul 17 15:23:12.449488 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:12.449468    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx_odh-ai-gateway-infra(a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb"
Jul 17 15:23:12.487633 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:12.487607    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:23:13.588238 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:13.588217    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:23:13.600256 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:13.600237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca\") pod \"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb\" (UID: \"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb\") "
Jul 17 15:23:13.600536 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:13.600517    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" (UID: "a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:23:13.701346 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:13.701314    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:23:14.461288 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:14.461258    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"
Jul 17 15:23:14.461433 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:14.461297    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx" event={"ID":"a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb","Type":"ContainerDied","Data":"f2607cd12072b6057a5f1639f9bce263013ebc14898d4d07166dd6bf8db3ffb1"}
Jul 17 15:23:14.461433 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:14.461384    2564 scope.go:117] "RemoveContainer" containerID="4317208dd31c5130c9d70a82653635aa9afc6e8268781b6bbe777fc37e773850"
Jul 17 15:23:14.492691 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:14.492659    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:23:14.495980 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:14.495959    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-gd9hx"]
Jul 17 15:23:15.950633 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:15.950598    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" path="/var/lib/kubelet/pods/a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb/volumes"
Jul 17 15:23:33.438261 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438219    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438606    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438619    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438639    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438644    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438659    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:23:33.438674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438665    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:23:33.438868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438720    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.438868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438728    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1053404b-1993-4836-82f7-d54fa6a8cb4e" containerName="revoke-keys"
Jul 17 15:23:33.438868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.438736    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:33.441056 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.441041    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:33.444212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.444188    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:23:33.444360 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.444232    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:23:33.452375 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.452351    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:33.552916 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.552889    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4\" (UID: \"5726e22b-2487-490c-8fa4-27b5df40f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:33.653735 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.653701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4\" (UID: \"5726e22b-2487-490c-8fa4-27b5df40f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:33.654342 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.654324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4\" (UID: \"5726e22b-2487-490c-8fa4-27b5df40f939\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:33.751569 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.751470    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:33.894871 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:33.894843    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:33.897537 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:23:33.897504    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5726e22b_2487_490c_8fa4_27b5df40f939.slice/crio-efb3f0ebb2fe1acc0fb1d3b6f6acb8089e83cf85bd546f054901a7c1f86cf7b6 WatchSource:0}: Error finding container efb3f0ebb2fe1acc0fb1d3b6f6acb8089e83cf85bd546f054901a7c1f86cf7b6: Status 404 returned error can't find the container with id efb3f0ebb2fe1acc0fb1d3b6f6acb8089e83cf85bd546f054901a7c1f86cf7b6
Jul 17 15:23:34.561152 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:34.561123    2564 generic.go:356] "Generic (PLEG): container finished" podID="5726e22b-2487-490c-8fa4-27b5df40f939" containerID="e039138007f5f55baffcb257149993383f197cd827004dc27d0b6d52a2cb2069" exitCode=6
Jul 17 15:23:34.561450 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:34.561214    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" event={"ID":"5726e22b-2487-490c-8fa4-27b5df40f939","Type":"ContainerDied","Data":"e039138007f5f55baffcb257149993383f197cd827004dc27d0b6d52a2cb2069"}
Jul 17 15:23:34.561450 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:34.561284    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" event={"ID":"5726e22b-2487-490c-8fa4-27b5df40f939","Type":"ContainerStarted","Data":"efb3f0ebb2fe1acc0fb1d3b6f6acb8089e83cf85bd546f054901a7c1f86cf7b6"}
Jul 17 15:23:34.561450 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:34.561395    2564 scope.go:117] "RemoveContainer" containerID="e039138007f5f55baffcb257149993383f197cd827004dc27d0b6d52a2cb2069"
Jul 17 15:23:35.568047 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:35.568008    2564 generic.go:356] "Generic (PLEG): container finished" podID="5726e22b-2487-490c-8fa4-27b5df40f939" containerID="780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970" exitCode=6
Jul 17 15:23:35.568469 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:35.568085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" event={"ID":"5726e22b-2487-490c-8fa4-27b5df40f939","Type":"ContainerDied","Data":"780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970"}
Jul 17 15:23:35.568469 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:35.568146    2564 scope.go:117] "RemoveContainer" containerID="e039138007f5f55baffcb257149993383f197cd827004dc27d0b6d52a2cb2069"
Jul 17 15:23:35.568469 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:35.568333    2564 scope.go:117] "RemoveContainer" containerID="780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970"
Jul 17 15:23:35.568587 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:35.568564    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4_odh-ai-gateway-infra(5726e22b-2487-490c-8fa4-27b5df40f939)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" podUID="5726e22b-2487-490c-8fa4-27b5df40f939"
Jul 17 15:23:36.574816 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:36.574791    2564 scope.go:117] "RemoveContainer" containerID="780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970"
Jul 17 15:23:36.575209 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:36.574983    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4_odh-ai-gateway-infra(5726e22b-2487-490c-8fa4-27b5df40f939)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" podUID="5726e22b-2487-490c-8fa4-27b5df40f939"
Jul 17 15:23:45.846431 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.846400    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:23:45.846940 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.846751    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:45.846940 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.846761    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:45.846940 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.846846    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6d309b0-2c9c-4496-8cbc-d4b9fbb2eebb" containerName="revoke-keys"
Jul 17 15:23:45.850244 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.850228    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:23:45.862225 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.862200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:23:45.959678 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:45.959648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-94lpn\" (UID: \"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:23:46.061209 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:46.061177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-94lpn\" (UID: \"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:23:46.061705 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:46.061685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-94lpn\" (UID: \"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:23:46.161406 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:46.161330    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:23:46.518390 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:46.518356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:23:46.520481 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:23:46.520442    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd636afb3_75d6_4bbf_9bc9_5f8fc0287a1a.slice/crio-21db347fabd040093d0dcd349f458a459e294e443ee4e254180de4b1db04b4a7 WatchSource:0}: Error finding container 21db347fabd040093d0dcd349f458a459e294e443ee4e254180de4b1db04b4a7: Status 404 returned error can't find the container with id 21db347fabd040093d0dcd349f458a459e294e443ee4e254180de4b1db04b4a7
Jul 17 15:23:46.623302 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:46.623267    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" event={"ID":"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a","Type":"ContainerStarted","Data":"21db347fabd040093d0dcd349f458a459e294e443ee4e254180de4b1db04b4a7"}
Jul 17 15:23:47.629798 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:47.629758    2564 generic.go:356] "Generic (PLEG): container finished" podID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerID="3a85c23e1a38cd9be8630f8d8382bedbf48fa8f24f1ccd2adf3b651bae9b3001" exitCode=7
Jul 17 15:23:47.629798 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:47.629792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" event={"ID":"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a","Type":"ContainerDied","Data":"3a85c23e1a38cd9be8630f8d8382bedbf48fa8f24f1ccd2adf3b651bae9b3001"}
Jul 17 15:23:47.630339 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:47.630012    2564 scope.go:117] "RemoveContainer" containerID="3a85c23e1a38cd9be8630f8d8382bedbf48fa8f24f1ccd2adf3b651bae9b3001"
Jul 17 15:23:48.637081 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:48.637037    2564 generic.go:356] "Generic (PLEG): container finished" podID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerID="59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9" exitCode=7
Jul 17 15:23:48.637551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:48.637092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" event={"ID":"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a","Type":"ContainerDied","Data":"59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9"}
Jul 17 15:23:48.637551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:48.637156    2564 scope.go:117] "RemoveContainer" containerID="3a85c23e1a38cd9be8630f8d8382bedbf48fa8f24f1ccd2adf3b651bae9b3001"
Jul 17 15:23:48.637551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:48.637317    2564 scope.go:117] "RemoveContainer" containerID="59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9"
Jul 17 15:23:48.637696 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:48.637555    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-94lpn_odh-ai-gateway-infra(d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a"
Jul 17 15:23:49.643911 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:49.643885    2564 scope.go:117] "RemoveContainer" containerID="59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9"
Jul 17 15:23:49.644339 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:49.644109    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-94lpn_odh-ai-gateway-infra(d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a"
Jul 17 15:23:50.944988 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:50.944956    2564 scope.go:117] "RemoveContainer" containerID="780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970"
Jul 17 15:23:51.655778 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:51.655743    2564 generic.go:356] "Generic (PLEG): container finished" podID="5726e22b-2487-490c-8fa4-27b5df40f939" containerID="a46fc0748f7bf137cf0e7ce2e63be4976d8467d47ea915e696a6ea8c79b74843" exitCode=6
Jul 17 15:23:51.655933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:51.655819    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" event={"ID":"5726e22b-2487-490c-8fa4-27b5df40f939","Type":"ContainerDied","Data":"a46fc0748f7bf137cf0e7ce2e63be4976d8467d47ea915e696a6ea8c79b74843"}
Jul 17 15:23:51.655933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:51.655880    2564 scope.go:117] "RemoveContainer" containerID="780b86d856d52d616c06898aef4070d10cf1e30cad714c65a5b2bbc5d0e4b970"
Jul 17 15:23:51.656189 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:51.656172    2564 scope.go:117] "RemoveContainer" containerID="a46fc0748f7bf137cf0e7ce2e63be4976d8467d47ea915e696a6ea8c79b74843"
Jul 17 15:23:51.656387 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:23:51.656368    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4_odh-ai-gateway-infra(5726e22b-2487-490c-8fa4-27b5df40f939)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" podUID="5726e22b-2487-490c-8fa4-27b5df40f939"
Jul 17 15:23:52.687684 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:52.687635    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:52.827174 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:52.827151    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:52.932180 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:52.932156    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca\") pod \"5726e22b-2487-490c-8fa4-27b5df40f939\" (UID: \"5726e22b-2487-490c-8fa4-27b5df40f939\") "
Jul 17 15:23:52.932489 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:52.932468    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5726e22b-2487-490c-8fa4-27b5df40f939" (UID: "5726e22b-2487-490c-8fa4-27b5df40f939"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:23:53.033574 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.033258    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5726e22b-2487-490c-8fa4-27b5df40f939-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:23:53.669253 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.669224    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"
Jul 17 15:23:53.669440 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.669263    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4" event={"ID":"5726e22b-2487-490c-8fa4-27b5df40f939","Type":"ContainerDied","Data":"efb3f0ebb2fe1acc0fb1d3b6f6acb8089e83cf85bd546f054901a7c1f86cf7b6"}
Jul 17 15:23:53.669440 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.669325    2564 scope.go:117] "RemoveContainer" containerID="a46fc0748f7bf137cf0e7ce2e63be4976d8467d47ea915e696a6ea8c79b74843"
Jul 17 15:23:53.692902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.692875    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:53.700197 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.700177    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f4d42d10-4fxr4"]
Jul 17 15:23:53.950725 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:23:53.950647    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" path="/var/lib/kubelet/pods/5726e22b-2487-490c-8fa4-27b5df40f939/volumes"
Jul 17 15:24:03.944780 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:03.944748    2564 scope.go:117] "RemoveContainer" containerID="59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9"
Jul 17 15:24:04.724965 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:04.724925    2564 generic.go:356] "Generic (PLEG): container finished" podID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerID="aae443fb1bd9f750215ec8f397c6d72c37be76742209ffd7f70a425211f41e93" exitCode=7
Jul 17 15:24:04.725140 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:04.725003    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" event={"ID":"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a","Type":"ContainerDied","Data":"aae443fb1bd9f750215ec8f397c6d72c37be76742209ffd7f70a425211f41e93"}
Jul 17 15:24:04.725140 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:04.725092    2564 scope.go:117] "RemoveContainer" containerID="59c8574477844851e72203d2d2c4b1366e6fb09a77150509d80272b89b28f9f9"
Jul 17 15:24:04.725332 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:04.725313    2564 scope.go:117] "RemoveContainer" containerID="aae443fb1bd9f750215ec8f397c6d72c37be76742209ffd7f70a425211f41e93"
Jul 17 15:24:04.725588 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:24:04.725565    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-94lpn_odh-ai-gateway-infra(d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a"
Jul 17 15:24:05.763700 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:05.763668    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:24:05.897920 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:05.897898    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:24:06.046536 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.046454    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca\") pod \"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a\" (UID: \"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a\") "
Jul 17 15:24:06.046879 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.046847    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" (UID: "d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:24:06.148046 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.148011    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:24:06.738714 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.738688    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"
Jul 17 15:24:06.738948 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.738730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn" event={"ID":"d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a","Type":"ContainerDied","Data":"21db347fabd040093d0dcd349f458a459e294e443ee4e254180de4b1db04b4a7"}
Jul 17 15:24:06.738948 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.738792    2564 scope.go:117] "RemoveContainer" containerID="aae443fb1bd9f750215ec8f397c6d72c37be76742209ffd7f70a425211f41e93"
Jul 17 15:24:06.761122 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.761097    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:24:06.765039 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:06.765015    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-94lpn"]
Jul 17 15:24:07.950056 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:07.950020    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" path="/var/lib/kubelet/pods/d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a/volumes"
Jul 17 15:24:45.929787 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.929711    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930128    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930144    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930158    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930163    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930172    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930177    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930185    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930191    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930259    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930268    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930270 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930275    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.930603 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930281    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:24:45.930603 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.930286    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:24:45.932589 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.932571    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:24:45.935463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.935442    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:24:45.935605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.935446    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:24:45.944215 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:45.944197    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:24:46.088650 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.088618    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm\" (UID: \"b707bfe1-53e3-490a-afe7-92d80ba71867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:24:46.190107 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.190008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm\" (UID: \"b707bfe1-53e3-490a-afe7-92d80ba71867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:24:46.190652 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.190632    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm\" (UID: \"b707bfe1-53e3-490a-afe7-92d80ba71867\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:24:46.241931 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.241902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:24:46.386273 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.386247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:24:46.387294 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:24:46.387270    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb707bfe1_53e3_490a_afe7_92d80ba71867.slice/crio-98b726ce64e1f5e8c782d5a1c1e89964c443f360922f6e0cee6e5581782d2b1a WatchSource:0}: Error finding container 98b726ce64e1f5e8c782d5a1c1e89964c443f360922f6e0cee6e5581782d2b1a: Status 404 returned error can't find the container with id 98b726ce64e1f5e8c782d5a1c1e89964c443f360922f6e0cee6e5581782d2b1a
Jul 17 15:24:46.935667 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.935580    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerStarted","Data":"0d358a5227d42cb1d000b8cd8db4da2f81506635b49e3627f2e627fd8b4fead2"}
Jul 17 15:24:46.935667 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.935636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerStarted","Data":"98b726ce64e1f5e8c782d5a1c1e89964c443f360922f6e0cee6e5581782d2b1a"}
Jul 17 15:24:46.961578 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:46.961535    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" podStartSLOduration=1.9615181769999999 podStartE2EDuration="1.961518177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:24:46.960080504 +0000 UTC m=+2097.527111633" watchObservedRunningTime="2026-07-17 15:24:46.961518177 +0000 UTC m=+2097.528549309"
Jul 17 15:24:47.941623 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:47.941588    2564 generic.go:356] "Generic (PLEG): container finished" podID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerID="0d358a5227d42cb1d000b8cd8db4da2f81506635b49e3627f2e627fd8b4fead2" exitCode=7
Jul 17 15:24:47.942000 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:47.941646    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerDied","Data":"0d358a5227d42cb1d000b8cd8db4da2f81506635b49e3627f2e627fd8b4fead2"}
Jul 17 15:24:47.942000 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:47.941883    2564 scope.go:117] "RemoveContainer" containerID="0d358a5227d42cb1d000b8cd8db4da2f81506635b49e3627f2e627fd8b4fead2"
Jul 17 15:24:48.949337 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:48.949301    2564 generic.go:356] "Generic (PLEG): container finished" podID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerID="897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d" exitCode=7
Jul 17 15:24:48.949731 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:48.949393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerDied","Data":"897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d"}
Jul 17 15:24:48.949731 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:48.949457    2564 scope.go:117] "RemoveContainer" containerID="0d358a5227d42cb1d000b8cd8db4da2f81506635b49e3627f2e627fd8b4fead2"
Jul 17 15:24:48.949731 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:48.949675    2564 scope.go:117] "RemoveContainer" containerID="897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d"
Jul 17 15:24:48.949894 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:24:48.949875    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm_odh-ai-gateway-infra(b707bfe1-53e3-490a-afe7-92d80ba71867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867"
Jul 17 15:24:49.957717 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:49.957691    2564 scope.go:117] "RemoveContainer" containerID="897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d"
Jul 17 15:24:49.958090 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:24:49.957853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm_odh-ai-gateway-infra(b707bfe1-53e3-490a-afe7-92d80ba71867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867"
Jul 17 15:24:50.020048 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:50.020011    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:24:50.024587 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:50.024566    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:24:50.025400 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:50.025372    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:24:50.030041 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:24:50.030027    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:25:04.947184 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:04.944317    2564 scope.go:117] "RemoveContainer" containerID="897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d"
Jul 17 15:25:06.032338 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:06.032304    2564 generic.go:356] "Generic (PLEG): container finished" podID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerID="824ee579adf60ea916cff3b555f8092ce7dbb02606b1ee4a06d75ab3221a32f8" exitCode=7
Jul 17 15:25:06.032842 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:06.032383    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerDied","Data":"824ee579adf60ea916cff3b555f8092ce7dbb02606b1ee4a06d75ab3221a32f8"}
Jul 17 15:25:06.032842 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:06.032443    2564 scope.go:117] "RemoveContainer" containerID="897a9184b9068eb88f080ebf04723da8456798018186466b94e0effc0fb6cd4d"
Jul 17 15:25:06.032842 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:06.032750    2564 scope.go:117] "RemoveContainer" containerID="824ee579adf60ea916cff3b555f8092ce7dbb02606b1ee4a06d75ab3221a32f8"
Jul 17 15:25:06.033022 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:06.032996    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm_odh-ai-gateway-infra(b707bfe1-53e3-490a-afe7-92d80ba71867)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867"
Jul 17 15:25:07.062742 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:07.062711    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:25:07.200287 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:07.200266    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:25:07.249979 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:07.249953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca\") pod \"b707bfe1-53e3-490a-afe7-92d80ba71867\" (UID: \"b707bfe1-53e3-490a-afe7-92d80ba71867\") "
Jul 17 15:25:07.250320 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:07.250301    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b707bfe1-53e3-490a-afe7-92d80ba71867" (UID: "b707bfe1-53e3-490a-afe7-92d80ba71867"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:25:07.351202 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:07.351119    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b707bfe1-53e3-490a-afe7-92d80ba71867-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:25:08.045904 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:08.045878    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"
Jul 17 15:25:08.046056 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:08.045914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm" event={"ID":"b707bfe1-53e3-490a-afe7-92d80ba71867","Type":"ContainerDied","Data":"98b726ce64e1f5e8c782d5a1c1e89964c443f360922f6e0cee6e5581782d2b1a"}
Jul 17 15:25:08.046056 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:08.045968    2564 scope.go:117] "RemoveContainer" containerID="824ee579adf60ea916cff3b555f8092ce7dbb02606b1ee4a06d75ab3221a32f8"
Jul 17 15:25:08.065940 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:08.065921    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:25:08.074476 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:08.074454    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c0e88560-7n6jm"]
Jul 17 15:25:09.950352 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:09.950321    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" path="/var/lib/kubelet/pods/b707bfe1-53e3-490a-afe7-92d80ba71867/volumes"
Jul 17 15:25:19.289151 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:19.289117    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:25:33.875404 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875365    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:25:33.875795 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875781    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875796    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875815    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:25:33.875839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d636afb3-75d6-4bbf-9bc9-5f8fc0287a1a" containerName="revoke-keys"
Jul 17 15:25:33.875839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875828    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875835    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875850    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875856    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875935    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875947    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875955    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:33.875993 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.875962    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5726e22b-2487-490c-8fa4-27b5df40f939" containerName="revoke-keys"
Jul 17 15:25:33.878497 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.878480    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:33.883379 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.883351    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:25:33.883500 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.883406    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:25:33.884289 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.884266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c65261cb-serving-cert\""
Jul 17 15:25:33.884414 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.884303    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:25:33.893615 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.893593    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:25:33.960139 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.960111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd6rx\" (UniqueName: \"kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:33.960342 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.960167    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:33.960342 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:33.960259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.061567 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.061530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.061748 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.061630    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.061748 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:34.061639    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-serving-cert: secret "maas-api-e2e-derive-c65261cb-serving-cert" not found
Jul 17 15:25:34.061748 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.061684    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd6rx\" (UniqueName: \"kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.061748 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:34.061709    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls podName:3865c2aa-0ede-4e16-a36f-380eccfa17ba nodeName:}" failed. No retries permitted until 2026-07-17 15:25:34.561688195 +0000 UTC m=+2145.128719305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls") pod "maas-api-e2e-derive-c65261cb-647bf8765-glzvs" (UID: "3865c2aa-0ede-4e16-a36f-380eccfa17ba") : secret "maas-api-e2e-derive-c65261cb-serving-cert" not found
Jul 17 15:25:34.062388 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.062370    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.072132 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.072114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd6rx\" (UniqueName: \"kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.567454 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.567416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.569884 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.569863    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") pod \"maas-api-e2e-derive-c65261cb-647bf8765-glzvs\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.788538 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.788502    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:34.928163 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.928135    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:25:34.930221 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:25:34.930188    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3865c2aa_0ede_4e16_a36f_380eccfa17ba.slice/crio-636ff6929975c4792ec9913995dee5a5e65164e0d6b7090a8f0e0fd7909e511e WatchSource:0}: Error finding container 636ff6929975c4792ec9913995dee5a5e65164e0d6b7090a8f0e0fd7909e511e: Status 404 returned error can't find the container with id 636ff6929975c4792ec9913995dee5a5e65164e0d6b7090a8f0e0fd7909e511e
Jul 17 15:25:34.931582 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:34.931566    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:25:35.180844 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:35.180761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerStarted","Data":"636ff6929975c4792ec9913995dee5a5e65164e0d6b7090a8f0e0fd7909e511e"}
Jul 17 15:25:36.690933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:36.690894    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:25:38.117106 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.117057    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:38.117584 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.117550    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:38.117584 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.117567    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b707bfe1-53e3-490a-afe7-92d80ba71867" containerName="revoke-keys"
Jul 17 15:25:38.119977 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.119956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:38.122708 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.122686    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:25:38.130736 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.130713    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:38.196999 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.196958    2564 generic.go:356] "Generic (PLEG): container finished" podID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerID="3ac67a9871abc436598857b72bd179f8597f2bd07ee3c75fedb50d2ef2c2900d" exitCode=1
Jul 17 15:25:38.197198 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.197029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"3ac67a9871abc436598857b72bd179f8597f2bd07ee3c75fedb50d2ef2c2900d"}
Jul 17 15:25:38.197329 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.197314    2564 scope.go:117] "RemoveContainer" containerID="3ac67a9871abc436598857b72bd179f8597f2bd07ee3c75fedb50d2ef2c2900d"
Jul 17 15:25:38.201419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.201393    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf\" (UID: \"2ec3db5c-5772-4364-9119-96ce327d42f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:38.302542 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.302507    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf\" (UID: \"2ec3db5c-5772-4364-9119-96ce327d42f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:38.303292 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.303271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf\" (UID: \"2ec3db5c-5772-4364-9119-96ce327d42f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:38.431519 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.431424    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:38.567683 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:38.567654    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:38.570182 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:25:38.570152    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ec3db5c_5772_4364_9119_96ce327d42f4.slice/crio-a08f9ea625783b22b46c30c6d81ebec81457c0b7c771681a6b2d524210c3bb24 WatchSource:0}: Error finding container a08f9ea625783b22b46c30c6d81ebec81457c0b7c771681a6b2d524210c3bb24: Status 404 returned error can't find the container with id a08f9ea625783b22b46c30c6d81ebec81457c0b7c771681a6b2d524210c3bb24
Jul 17 15:25:39.203594 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.203562    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerID="3662810b3b0637c93b10d0fa257dccd2c7441d912bcc4ea27f16b4ee4aab17d1" exitCode=7
Jul 17 15:25:39.204115 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.203648    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerDied","Data":"3662810b3b0637c93b10d0fa257dccd2c7441d912bcc4ea27f16b4ee4aab17d1"}
Jul 17 15:25:39.204115 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.203712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerStarted","Data":"a08f9ea625783b22b46c30c6d81ebec81457c0b7c771681a6b2d524210c3bb24"}
Jul 17 15:25:39.204115 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.203920    2564 scope.go:117] "RemoveContainer" containerID="3662810b3b0637c93b10d0fa257dccd2c7441d912bcc4ea27f16b4ee4aab17d1"
Jul 17 15:25:39.205736 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.205711    2564 generic.go:356] "Generic (PLEG): container finished" podID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b" exitCode=1
Jul 17 15:25:39.205841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.205796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"}
Jul 17 15:25:39.205902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.205844    2564 scope.go:117] "RemoveContainer" containerID="3ac67a9871abc436598857b72bd179f8597f2bd07ee3c75fedb50d2ef2c2900d"
Jul 17 15:25:39.206059 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.206041    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:25:39.206276 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:39.206252    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:25:39.788857 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:39.788824    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:40.211989 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:40.211961    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:25:40.212491 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:40.212197    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:25:40.213631 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:40.213607    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerID="bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a" exitCode=7
Jul 17 15:25:40.213768 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:40.213636    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerDied","Data":"bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a"}
Jul 17 15:25:40.213768 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:40.213674    2564 scope.go:117] "RemoveContainer" containerID="3662810b3b0637c93b10d0fa257dccd2c7441d912bcc4ea27f16b4ee4aab17d1"
Jul 17 15:25:40.213892 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:40.213858    2564 scope.go:117] "RemoveContainer" containerID="bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a"
Jul 17 15:25:40.214095 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:40.214041    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf_odh-ai-gateway-infra(2ec3db5c-5772-4364-9119-96ce327d42f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4"
Jul 17 15:25:41.220258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:41.220230    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:25:41.220668 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:41.220309    2564 scope.go:117] "RemoveContainer" containerID="bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a"
Jul 17 15:25:41.220668 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:41.220409    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:25:41.220668 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:41.220488    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf_odh-ai-gateway-infra(2ec3db5c-5772-4364-9119-96ce327d42f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4"
Jul 17 15:25:44.789428 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:44.789389    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:25:44.789810 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:44.789799    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:25:44.789987 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:44.789968    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:25:54.944718 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:54.944686    2564 scope.go:117] "RemoveContainer" containerID="bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a"
Jul 17 15:25:55.296048 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:55.296004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerStarted","Data":"d98afa67f8779727ab3e5f3cb6bf6b62566531d7cbccbb1eb282b9de7df9774a"}
Jul 17 15:25:55.320783 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:55.320734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" podStartSLOduration=17.320719566 podStartE2EDuration="17.320719566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:25:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:25:55.318574881 +0000 UTC m=+2165.885606013" watchObservedRunningTime="2026-07-17 15:25:55.320719566 +0000 UTC m=+2165.887750698"
Jul 17 15:25:56.302653 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:56.302619    2564 generic.go:356] "Generic (PLEG): container finished" podID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerID="d98afa67f8779727ab3e5f3cb6bf6b62566531d7cbccbb1eb282b9de7df9774a" exitCode=7
Jul 17 15:25:56.303116 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:56.302691    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerDied","Data":"d98afa67f8779727ab3e5f3cb6bf6b62566531d7cbccbb1eb282b9de7df9774a"}
Jul 17 15:25:56.303116 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:56.302764    2564 scope.go:117] "RemoveContainer" containerID="bbc71dd40160083cd9f75e63fddf649859f04be0efb42c09e3221a984293345a"
Jul 17 15:25:56.303116 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:56.303052    2564 scope.go:117] "RemoveContainer" containerID="d98afa67f8779727ab3e5f3cb6bf6b62566531d7cbccbb1eb282b9de7df9774a"
Jul 17 15:25:56.303323 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:25:56.303298    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf_odh-ai-gateway-infra(2ec3db5c-5772-4364-9119-96ce327d42f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4"
Jul 17 15:25:56.331319 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:56.331294    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:57.446868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:57.446846    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:57.575015 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:57.574989    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca\") pod \"2ec3db5c-5772-4364-9119-96ce327d42f4\" (UID: \"2ec3db5c-5772-4364-9119-96ce327d42f4\") "
Jul 17 15:25:57.575413 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:57.575392    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ec3db5c-5772-4364-9119-96ce327d42f4" (UID: "2ec3db5c-5772-4364-9119-96ce327d42f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:25:57.675996 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:57.675969    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ec3db5c-5772-4364-9119-96ce327d42f4-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:25:58.314205 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.314182    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"
Jul 17 15:25:58.314393 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.314216    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf" event={"ID":"2ec3db5c-5772-4364-9119-96ce327d42f4","Type":"ContainerDied","Data":"a08f9ea625783b22b46c30c6d81ebec81457c0b7c771681a6b2d524210c3bb24"}
Jul 17 15:25:58.314393 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.314281    2564 scope.go:117] "RemoveContainer" containerID="d98afa67f8779727ab3e5f3cb6bf6b62566531d7cbccbb1eb282b9de7df9774a"
Jul 17 15:25:58.364372 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.364339    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:58.372321 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.372300    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-x5qdf"]
Jul 17 15:25:58.945084 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:58.945030    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:25:59.950153 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:25:59.950054    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" path="/var/lib/kubelet/pods/2ec3db5c-5772-4364-9119-96ce327d42f4/volumes"
Jul 17 15:26:00.328207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:00.328177    2564 generic.go:356] "Generic (PLEG): container finished" podID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d" exitCode=1
Jul 17 15:26:00.328371 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:00.328254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"}
Jul 17 15:26:00.328371 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:00.328317    2564 scope.go:117] "RemoveContainer" containerID="c36ac94b7f9b7176407be728502470d77f8826b3dc316605d30614073e69554b"
Jul 17 15:26:00.328653 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:00.328637    2564 scope.go:117] "RemoveContainer" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"
Jul 17 15:26:00.328819 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:00.328797    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:04.788783 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:04.788744    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:26:04.789183 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:04.788813    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:26:04.789183 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:04.789143    2564 scope.go:117] "RemoveContainer" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"
Jul 17 15:26:04.789315 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:04.789297    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:18.944256 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:18.944219    2564 scope.go:117] "RemoveContainer" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"
Jul 17 15:26:18.944724 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:18.944445    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:33.944798 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:33.944758    2564 scope.go:117] "RemoveContainer" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"
Jul 17 15:26:34.499756 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:34.499671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerStarted","Data":"8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"}
Jul 17 15:26:34.499909 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:34.499893    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:26:34.525403 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:34.525353    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podStartSLOduration=58.233455096 podStartE2EDuration="1m1.525339081s" totalImagesPullingTime="3.291883985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:25:33 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:26:34.523341822 +0000 UTC m=+2205.090372964" watchObservedRunningTime="2026-07-17 15:26:34.525339081 +0000 UTC m=+2205.092370212"
Jul 17 15:26:35.509584 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:35.509547    2564 generic.go:356] "Generic (PLEG): container finished" podID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c" exitCode=1
Jul 17 15:26:35.510032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:35.509629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"}
Jul 17 15:26:35.510754 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:35.509697    2564 scope.go:117] "RemoveContainer" containerID="94d98ac28986ced27c3eaa7c693abc53971067519f26c088bcf20a1ca5c0191d"
Jul 17 15:26:35.510832 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:35.510820    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:26:35.511031 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:35.511014    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:36.516445 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:36.516409    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:26:36.516922 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:36.516644    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:38.206573 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206539    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206909    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206920    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206933    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206939    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206948    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.206952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.206953    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.207191 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.207012    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.207191 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.207019    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:26:38.210097 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.210057    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:38.213081 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.213044    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:26:38.221603 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.221582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:38.311585 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.311549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8\" (UID: \"89ca4478-77ea-4a81-8de7-ec8ef07578ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:38.412551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.412519    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8\" (UID: \"89ca4478-77ea-4a81-8de7-ec8ef07578ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:38.413218 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.413197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8\" (UID: \"89ca4478-77ea-4a81-8de7-ec8ef07578ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:38.520315 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.520225    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:38.666755 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:38.666731    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:38.668339 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:26:38.668311    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ca4478_77ea_4a81_8de7_ec8ef07578ec.slice/crio-a673b6b4041ae08b22626c8ab77436c53df76bfe93ff1c2bb0adc50e23130c14 WatchSource:0}: Error finding container a673b6b4041ae08b22626c8ab77436c53df76bfe93ff1c2bb0adc50e23130c14: Status 404 returned error can't find the container with id a673b6b4041ae08b22626c8ab77436c53df76bfe93ff1c2bb0adc50e23130c14
Jul 17 15:26:39.532329 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:39.532246    2564 generic.go:356] "Generic (PLEG): container finished" podID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerID="63e0a56a9034100fe96bf8d064de8f00c022361289a3e1afecf7e8bd595bd9a6" exitCode=7
Jul 17 15:26:39.532683 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:39.532332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" event={"ID":"89ca4478-77ea-4a81-8de7-ec8ef07578ec","Type":"ContainerDied","Data":"63e0a56a9034100fe96bf8d064de8f00c022361289a3e1afecf7e8bd595bd9a6"}
Jul 17 15:26:39.532683 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:39.532390    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" event={"ID":"89ca4478-77ea-4a81-8de7-ec8ef07578ec","Type":"ContainerStarted","Data":"a673b6b4041ae08b22626c8ab77436c53df76bfe93ff1c2bb0adc50e23130c14"}
Jul 17 15:26:39.532683 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:39.532502    2564 scope.go:117] "RemoveContainer" containerID="63e0a56a9034100fe96bf8d064de8f00c022361289a3e1afecf7e8bd595bd9a6"
Jul 17 15:26:40.539762 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:40.539673    2564 generic.go:356] "Generic (PLEG): container finished" podID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerID="8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef" exitCode=7
Jul 17 15:26:40.540223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:40.539748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" event={"ID":"89ca4478-77ea-4a81-8de7-ec8ef07578ec","Type":"ContainerDied","Data":"8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef"}
Jul 17 15:26:40.540223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:40.539817    2564 scope.go:117] "RemoveContainer" containerID="63e0a56a9034100fe96bf8d064de8f00c022361289a3e1afecf7e8bd595bd9a6"
Jul 17 15:26:40.540223 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:40.540079    2564 scope.go:117] "RemoveContainer" containerID="8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef"
Jul 17 15:26:40.540350 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:40.540283    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8_odh-ai-gateway-infra(89ca4478-77ea-4a81-8de7-ec8ef07578ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec"
Jul 17 15:26:41.546209 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:41.546175    2564 scope.go:117] "RemoveContainer" containerID="8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef"
Jul 17 15:26:41.546568 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:41.546363    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8_odh-ai-gateway-infra(89ca4478-77ea-4a81-8de7-ec8ef07578ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec"
Jul 17 15:26:44.789211 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:44.789175    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:26:44.789606 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:44.789591    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:26:44.789785 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:44.789766    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:26:51.944850 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:51.944819    2564 scope.go:117] "RemoveContainer" containerID="8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef"
Jul 17 15:26:52.602489 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:52.602461    2564 generic.go:356] "Generic (PLEG): container finished" podID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerID="9630580a7e2038dabf39aca205fb4daf112ac027f6bc4b4c5cc866f8d1f5174b" exitCode=7
Jul 17 15:26:52.602628 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:52.602539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" event={"ID":"89ca4478-77ea-4a81-8de7-ec8ef07578ec","Type":"ContainerDied","Data":"9630580a7e2038dabf39aca205fb4daf112ac027f6bc4b4c5cc866f8d1f5174b"}
Jul 17 15:26:52.602628 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:52.602604    2564 scope.go:117] "RemoveContainer" containerID="8e88b1a8b0bb616dfd8856e2ffd932001ad9f6532dea668af132163c856af0ef"
Jul 17 15:26:52.602933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:52.602915    2564 scope.go:117] "RemoveContainer" containerID="9630580a7e2038dabf39aca205fb4daf112ac027f6bc4b4c5cc866f8d1f5174b"
Jul 17 15:26:52.603168 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:52.603147    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8_odh-ai-gateway-infra(89ca4478-77ea-4a81-8de7-ec8ef07578ec)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec"
Jul 17 15:26:53.651811 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:53.651778    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:53.790660 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:53.790638    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:53.938463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:53.938376    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca\") pod \"89ca4478-77ea-4a81-8de7-ec8ef07578ec\" (UID: \"89ca4478-77ea-4a81-8de7-ec8ef07578ec\") "
Jul 17 15:26:53.938729 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:53.938707    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89ca4478-77ea-4a81-8de7-ec8ef07578ec" (UID: "89ca4478-77ea-4a81-8de7-ec8ef07578ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:26:54.039436 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.039404    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ca4478-77ea-4a81-8de7-ec8ef07578ec-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:26:54.615046 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.615012    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"
Jul 17 15:26:54.615266 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.615019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8" event={"ID":"89ca4478-77ea-4a81-8de7-ec8ef07578ec","Type":"ContainerDied","Data":"a673b6b4041ae08b22626c8ab77436c53df76bfe93ff1c2bb0adc50e23130c14"}
Jul 17 15:26:54.615266 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.615153    2564 scope.go:117] "RemoveContainer" containerID="9630580a7e2038dabf39aca205fb4daf112ac027f6bc4b4c5cc866f8d1f5174b"
Jul 17 15:26:54.638259 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.638234    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:54.640724 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:54.640692    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-tm4c8"]
Jul 17 15:26:55.949968 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:55.949937    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" path="/var/lib/kubelet/pods/89ca4478-77ea-4a81-8de7-ec8ef07578ec/volumes"
Jul 17 15:26:57.944680 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:26:57.944640    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:26:57.945091 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:26:57.944800    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:12.944839 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:12.944762    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:27:12.945262 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:12.944938    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:27.945191 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:27.945156    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:27:28.272205 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272124    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:28.272524 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272510    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.272575 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272526    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.272575 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272537    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.272575 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272542    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.272676 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272612    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.272676 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272625    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ec3db5c-5772-4364-9119-96ce327d42f4" containerName="revoke-keys"
Jul 17 15:27:28.272676 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.272634    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:27:28.275150 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.275135    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:28.278060 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.278032    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:27:28.285084 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.285036    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:28.341207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.341178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc\" (UID: \"422cc1da-d68b-4070-9b49-36da329c02e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:28.441633 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.441614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc\" (UID: \"422cc1da-d68b-4070-9b49-36da329c02e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:28.442433 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.442412    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc\" (UID: \"422cc1da-d68b-4070-9b49-36da329c02e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:28.586292 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.586263    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:28.719401 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.719378    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:28.721711 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:27:28.721684    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422cc1da_d68b_4070_9b49_36da329c02e7.slice/crio-bf416aad1ef789eae45689bf0b488a049b64a09b81d7fee449c2471963521c1f WatchSource:0}: Error finding container bf416aad1ef789eae45689bf0b488a049b64a09b81d7fee449c2471963521c1f: Status 404 returned error can't find the container with id bf416aad1ef789eae45689bf0b488a049b64a09b81d7fee449c2471963521c1f
Jul 17 15:27:28.780751 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.780727    2564 generic.go:356] "Generic (PLEG): container finished" podID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50" exitCode=1
Jul 17 15:27:28.780848 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.780796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"}
Jul 17 15:27:28.780904 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.780853    2564 scope.go:117] "RemoveContainer" containerID="8ab8b64137ba6bbfec4557a981f876ced6583f97b4bbb4d33091fb58f2206e3c"
Jul 17 15:27:28.781225 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.781197    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:28.781432 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:28.781413    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:28.782094 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:28.782054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" event={"ID":"422cc1da-d68b-4070-9b49-36da329c02e7","Type":"ContainerStarted","Data":"bf416aad1ef789eae45689bf0b488a049b64a09b81d7fee449c2471963521c1f"}
Jul 17 15:27:29.500288 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:29.500255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:27:29.788292 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:29.788210    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:29.788448 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:29.788400    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:29.789477 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:29.789450    2564 generic.go:356] "Generic (PLEG): container finished" podID="422cc1da-d68b-4070-9b49-36da329c02e7" containerID="8497569f194742078b75b2bf58a19ab33de5b569e00ab3f059529f7d4b3793ac" exitCode=7
Jul 17 15:27:29.789576 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:29.789495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" event={"ID":"422cc1da-d68b-4070-9b49-36da329c02e7","Type":"ContainerDied","Data":"8497569f194742078b75b2bf58a19ab33de5b569e00ab3f059529f7d4b3793ac"}
Jul 17 15:27:29.789705 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:29.789693    2564 scope.go:117] "RemoveContainer" containerID="8497569f194742078b75b2bf58a19ab33de5b569e00ab3f059529f7d4b3793ac"
Jul 17 15:27:30.795804 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:30.795715    2564 generic.go:356] "Generic (PLEG): container finished" podID="422cc1da-d68b-4070-9b49-36da329c02e7" containerID="7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb" exitCode=7
Jul 17 15:27:30.795804 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:30.795767    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" event={"ID":"422cc1da-d68b-4070-9b49-36da329c02e7","Type":"ContainerDied","Data":"7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb"}
Jul 17 15:27:30.796331 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:30.795823    2564 scope.go:117] "RemoveContainer" containerID="8497569f194742078b75b2bf58a19ab33de5b569e00ab3f059529f7d4b3793ac"
Jul 17 15:27:30.796331 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:30.796048    2564 scope.go:117] "RemoveContainer" containerID="7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb"
Jul 17 15:27:30.796331 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:30.796313    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc_odh-ai-gateway-infra(422cc1da-d68b-4070-9b49-36da329c02e7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" podUID="422cc1da-d68b-4070-9b49-36da329c02e7"
Jul 17 15:27:31.802452 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:31.802424    2564 scope.go:117] "RemoveContainer" containerID="7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb"
Jul 17 15:27:31.802890 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:31.802673    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc_odh-ai-gateway-infra(422cc1da-d68b-4070-9b49-36da329c02e7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" podUID="422cc1da-d68b-4070-9b49-36da329c02e7"
Jul 17 15:27:34.788627 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:34.788587    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:27:34.789151 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:34.789120    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:34.789371 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:34.789346    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:45.945033 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:45.945003    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:45.945521 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:45.945048    2564 scope.go:117] "RemoveContainer" containerID="7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb"
Jul 17 15:27:45.945521 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:45.945229    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:46.881730 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:46.881695    2564 generic.go:356] "Generic (PLEG): container finished" podID="422cc1da-d68b-4070-9b49-36da329c02e7" containerID="a0976dc59270092efa25c3f010c9507c729bb9b9cbf1c8b0f6c861aa19778a6c" exitCode=7
Jul 17 15:27:46.881902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:46.881768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" event={"ID":"422cc1da-d68b-4070-9b49-36da329c02e7","Type":"ContainerDied","Data":"a0976dc59270092efa25c3f010c9507c729bb9b9cbf1c8b0f6c861aa19778a6c"}
Jul 17 15:27:46.881902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:46.881829    2564 scope.go:117] "RemoveContainer" containerID="7c2f5fe7083016b7bf55d63591ccdfb662248289604b9211be47b886656794fb"
Jul 17 15:27:46.882162 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:46.882145    2564 scope.go:117] "RemoveContainer" containerID="a0976dc59270092efa25c3f010c9507c729bb9b9cbf1c8b0f6c861aa19778a6c"
Jul 17 15:27:46.882375 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:46.882352    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc_odh-ai-gateway-infra(422cc1da-d68b-4070-9b49-36da329c02e7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" podUID="422cc1da-d68b-4070-9b49-36da329c02e7"
Jul 17 15:27:47.928659 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:47.928625    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:48.066044 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.066020    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:48.215014 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.214930    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca\") pod \"422cc1da-d68b-4070-9b49-36da329c02e7\" (UID: \"422cc1da-d68b-4070-9b49-36da329c02e7\") "
Jul 17 15:27:48.215283 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.215259    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "422cc1da-d68b-4070-9b49-36da329c02e7" (UID: "422cc1da-d68b-4070-9b49-36da329c02e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:27:48.316088 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.316030    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/422cc1da-d68b-4070-9b49-36da329c02e7-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:27:48.895112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.895052    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"
Jul 17 15:27:48.895289 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.895086    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc" event={"ID":"422cc1da-d68b-4070-9b49-36da329c02e7","Type":"ContainerDied","Data":"bf416aad1ef789eae45689bf0b488a049b64a09b81d7fee449c2471963521c1f"}
Jul 17 15:27:48.895289 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.895211    2564 scope.go:117] "RemoveContainer" containerID="a0976dc59270092efa25c3f010c9507c729bb9b9cbf1c8b0f6c861aa19778a6c"
Jul 17 15:27:48.926348 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.926323    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:48.927919 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:48.927897    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c65261cb-rr5sc"]
Jul 17 15:27:49.950945 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:49.950910    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" path="/var/lib/kubelet/pods/422cc1da-d68b-4070-9b49-36da329c02e7/volumes"
Jul 17 15:27:57.949881 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:57.949849    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:57.950293 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:27:57.950029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-derive-c65261cb-647bf8765-glzvs_odh-ai-gateway-infra(3865c2aa-0ede-4e16-a36f-380eccfa17ba)\"" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba"
Jul 17 15:27:59.009048 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.009008    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:27:59.151352 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.151326    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:27:59.309924 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.309841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") pod \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") "
Jul 17 15:27:59.309924 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.309900    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca\") pod \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") "
Jul 17 15:27:59.310130 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.309937    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd6rx\" (UniqueName: \"kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx\") pod \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\" (UID: \"3865c2aa-0ede-4e16-a36f-380eccfa17ba\") "
Jul 17 15:27:59.310310 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.310281    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3865c2aa-0ede-4e16-a36f-380eccfa17ba" (UID: "3865c2aa-0ede-4e16-a36f-380eccfa17ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:27:59.312132 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.312110    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx" (OuterVolumeSpecName: "kube-api-access-fd6rx") pod "3865c2aa-0ede-4e16-a36f-380eccfa17ba" (UID: "3865c2aa-0ede-4e16-a36f-380eccfa17ba"). InnerVolumeSpecName "kube-api-access-fd6rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:27:59.312226 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.312152    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3865c2aa-0ede-4e16-a36f-380eccfa17ba" (UID: "3865c2aa-0ede-4e16-a36f-380eccfa17ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:27:59.410608 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.410580    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3865c2aa-0ede-4e16-a36f-380eccfa17ba-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:27:59.410608 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.410604    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3865c2aa-0ede-4e16-a36f-380eccfa17ba-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:27:59.410608 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.410613    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd6rx\" (UniqueName: \"kubernetes.io/projected/3865c2aa-0ede-4e16-a36f-380eccfa17ba-kube-api-access-fd6rx\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:27:59.948026 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.947993    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"
Jul 17 15:27:59.952389 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.952356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs" event={"ID":"3865c2aa-0ede-4e16-a36f-380eccfa17ba","Type":"ContainerDied","Data":"636ff6929975c4792ec9913995dee5a5e65164e0d6b7090a8f0e0fd7909e511e"}
Jul 17 15:27:59.952519 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.952416    2564 scope.go:117] "RemoveContainer" containerID="f9c34e45eafef987e92819173085aa2f36c3af758eb900286c64d8db55e37c50"
Jul 17 15:27:59.985264 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.985234    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:27:59.990523 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:27:59.990497    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c65261cb-647bf8765-glzvs"]
Jul 17 15:28:01.930422 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:28:01.930388    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:28:01.950260 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:28:01.950228    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" path="/var/lib/kubelet/pods/3865c2aa-0ede-4e16-a36f-380eccfa17ba/volumes"
Jul 17 15:29:42.699707 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:42.699672    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:29:50.049658 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:50.049628    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:29:50.055301 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:50.055279    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:29:50.056605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:50.056581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:29:50.068383 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:50.068361    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:29:52.086750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:29:52.086712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:30:00.135140 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135094    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"]
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135453    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135464    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135472    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135477    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135485    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135491    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135498    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135503    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135509    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135515    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135525    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135530    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135542    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.135551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135547    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135598    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135605    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135610    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ca4478-77ea-4a81-8de7-ec8ef07578ec" containerName="revoke-keys"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135616    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135620    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135625    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="422cc1da-d68b-4070-9b49-36da329c02e7" containerName="revoke-keys"
Jul 17 15:30:00.136091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.135631    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.137944 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.137926    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"
Jul 17 15:30:00.140467 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.140450    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:30:00.147520 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.147498    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"]
Jul 17 15:30:00.148139 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.148125    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"
Jul 17 15:30:00.312840 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.312648    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"]
Jul 17 15:30:00.316903 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:30:00.316870    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0e62dd_690e_4e47_bdde_b5669a36839f.slice/crio-4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566 WatchSource:0}: Error finding container 4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566: Status 404 returned error can't find the container with id 4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566
Jul 17 15:30:00.417221 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.417154    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:00.418509 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418476    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.418509 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418503    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.418641 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418525    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.418641 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418535    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.418712 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418642    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.418712 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.418656    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3865c2aa-0ede-4e16-a36f-380eccfa17ba" containerName="maas-api"
Jul 17 15:30:00.421446 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.421426    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.428053 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.428031    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:30:00.428185 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.428139    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:30:00.428451 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.428432    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:30:00.428543 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.428440    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-afffcc69-serving-cert\""
Jul 17 15:30:00.451814 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.451777    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:00.529527 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.529501    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk89p\" (UniqueName: \"kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.529737 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.529564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.529737 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.529591    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.534337 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.534310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc" event={"ID":"bd0e62dd-690e-4e47-bdde-b5669a36839f","Type":"ContainerStarted","Data":"4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566"}
Jul 17 15:30:00.630279 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.630251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.630453 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.630287    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.630453 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.630375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk89p\" (UniqueName: \"kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.630940 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.630920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.632605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.632588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.650819 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.650800    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk89p\" (UniqueName: \"kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p\") pod \"maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:00.733765 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:00.733688    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:01.075731 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:01.075707    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:01.076812 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:30:01.076789    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f07a98_c706_4706_9dad_fcc894fe40c3.slice/crio-30c7850cba066f87a212df49697751087d48cb059c4a5bb425b2f4e865025386 WatchSource:0}: Error finding container 30c7850cba066f87a212df49697751087d48cb059c4a5bb425b2f4e865025386: Status 404 returned error can't find the container with id 30c7850cba066f87a212df49697751087d48cb059c4a5bb425b2f4e865025386
Jul 17 15:30:01.541477 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:01.541431    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" event={"ID":"08f07a98-c706-4706-9dad-fcc894fe40c3","Type":"ContainerStarted","Data":"30c7850cba066f87a212df49697751087d48cb059c4a5bb425b2f4e865025386"}
Jul 17 15:30:01.543015 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:01.542986    2564 generic.go:356] "Generic (PLEG): container finished" podID="bd0e62dd-690e-4e47-bdde-b5669a36839f" containerID="db69571e60d8476a37c58a0e1ff5356399204bc2292bda2416ca7c0c3d38504d" exitCode=0
Jul 17 15:30:01.543181 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:01.543090    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc" event={"ID":"bd0e62dd-690e-4e47-bdde-b5669a36839f","Type":"ContainerDied","Data":"db69571e60d8476a37c58a0e1ff5356399204bc2292bda2416ca7c0c3d38504d"}
Jul 17 15:30:02.549891 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:02.549790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" event={"ID":"08f07a98-c706-4706-9dad-fcc894fe40c3","Type":"ContainerStarted","Data":"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"}
Jul 17 15:30:02.550300 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:02.550077    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:02.572230 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:02.572165    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" podStartSLOduration=1.584252927 podStartE2EDuration="2.572143678s" totalImagesPullingTime="987.890751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:30:02.56940078 +0000 UTC m=+2413.136431912" watchObservedRunningTime="2026-07-17 15:30:02.572143678 +0000 UTC m=+2413.139174810"
Jul 17 15:30:02.684076 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:02.684036    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"
Jul 17 15:30:03.556255 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:03.556217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc" event={"ID":"bd0e62dd-690e-4e47-bdde-b5669a36839f","Type":"ContainerDied","Data":"4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566"}
Jul 17 15:30:03.556639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:03.556282    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f4630a591f96d7f128c6ec3c045f566a22550b8a1f5bd9e3a478c7f65819566"
Jul 17 15:30:03.556639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:03.556233    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738370-xhxbc"
Jul 17 15:30:05.491832 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:05.491797    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:30:08.561658 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:08.561630    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:12.843271 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.843190    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"]
Jul 17 15:30:12.843662 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.843569    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0e62dd-690e-4e47-bdde-b5669a36839f" containerName="cleanup"
Jul 17 15:30:12.843662 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.843581    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0e62dd-690e-4e47-bdde-b5669a36839f" containerName="cleanup"
Jul 17 15:30:12.843662 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.843639    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0e62dd-690e-4e47-bdde-b5669a36839f" containerName="cleanup"
Jul 17 15:30:12.845950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.845931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:12.848977 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.848956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:30:12.860857 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.860834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"]
Jul 17 15:30:12.942454 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:12.940967    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx\" (UID: \"72cba91e-74f3-4b93-b009-cd5755be1353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:13.042200 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:13.042164    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx\" (UID: \"72cba91e-74f3-4b93-b009-cd5755be1353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:13.042897 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:13.042868    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx\" (UID: \"72cba91e-74f3-4b93-b009-cd5755be1353\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:13.156183 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:13.156106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:13.296886 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:13.296796    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"]
Jul 17 15:30:13.299349 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:30:13.299320    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72cba91e_74f3_4b93_b009_cd5755be1353.slice/crio-45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762 WatchSource:0}: Error finding container 45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762: Status 404 returned error can't find the container with id 45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762
Jul 17 15:30:13.605420 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:13.605384    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx" event={"ID":"72cba91e-74f3-4b93-b009-cd5755be1353","Type":"ContainerStarted","Data":"45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762"}
Jul 17 15:30:14.611913 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:14.611878    2564 generic.go:356] "Generic (PLEG): container finished" podID="72cba91e-74f3-4b93-b009-cd5755be1353" containerID="1954be358c01f469874fa97a4af72f9b437e00d671412ba560b0b22e5d265cde" exitCode=0
Jul 17 15:30:14.612398 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:14.611966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx" event={"ID":"72cba91e-74f3-4b93-b009-cd5755be1353","Type":"ContainerDied","Data":"1954be358c01f469874fa97a4af72f9b437e00d671412ba560b0b22e5d265cde"}
Jul 17 15:30:15.750020 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:15.749999    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:15.869540 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:15.869463    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca\") pod \"72cba91e-74f3-4b93-b009-cd5755be1353\" (UID: \"72cba91e-74f3-4b93-b009-cd5755be1353\") "
Jul 17 15:30:15.869786 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:15.869765    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72cba91e-74f3-4b93-b009-cd5755be1353" (UID: "72cba91e-74f3-4b93-b009-cd5755be1353"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:30:15.971046 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:15.971020    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72cba91e-74f3-4b93-b009-cd5755be1353-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:30:16.623553 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:16.623523    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx"
Jul 17 15:30:16.623553 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:16.623547    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx" event={"ID":"72cba91e-74f3-4b93-b009-cd5755be1353","Type":"ContainerDied","Data":"45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762"}
Jul 17 15:30:16.623826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:16.623593    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45e6ce2300b0418e9e8ba0ba435dda17c3b711ae0ea6099362ed087272560762"
Jul 17 15:30:23.390441 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.390408    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:23.390927 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.390642    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerName="maas-api" containerID="cri-o://6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478" gracePeriod=30
Jul 17 15:30:23.557636 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.557597    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.63:8443/health\": dial tcp 10.133.0.63:8443: connect: connection refused"
Jul 17 15:30:23.653367 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.653299    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:23.658185 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.658157    2564 generic.go:356] "Generic (PLEG): container finished" podID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerID="6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478" exitCode=0
Jul 17 15:30:23.658322 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.658192    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"
Jul 17 15:30:23.658322 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.658233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" event={"ID":"08f07a98-c706-4706-9dad-fcc894fe40c3","Type":"ContainerDied","Data":"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"}
Jul 17 15:30:23.658322 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.658281    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7" event={"ID":"08f07a98-c706-4706-9dad-fcc894fe40c3","Type":"ContainerDied","Data":"30c7850cba066f87a212df49697751087d48cb059c4a5bb425b2f4e865025386"}
Jul 17 15:30:23.658322 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.658309    2564 scope.go:117] "RemoveContainer" containerID="6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"
Jul 17 15:30:23.666479 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.666461    2564 scope.go:117] "RemoveContainer" containerID="6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"
Jul 17 15:30:23.666757 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:30:23.666739    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478\": container with ID starting with 6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478 not found: ID does not exist" containerID="6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"
Jul 17 15:30:23.666817 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.666768    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478"} err="failed to get container status \"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478\": rpc error: code = NotFound desc = could not find container \"6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478\": container with ID starting with 6604d6e7ce37c60a05fd5b3c83e77f7af209e9306c369ed40ffb0f4de7c3e478 not found: ID does not exist"
Jul 17 15:30:23.735430 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.735401    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk89p\" (UniqueName: \"kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p\") pod \"08f07a98-c706-4706-9dad-fcc894fe40c3\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") "
Jul 17 15:30:23.735602 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.735449    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca\") pod \"08f07a98-c706-4706-9dad-fcc894fe40c3\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") "
Jul 17 15:30:23.735602 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.735481    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls\") pod \"08f07a98-c706-4706-9dad-fcc894fe40c3\" (UID: \"08f07a98-c706-4706-9dad-fcc894fe40c3\") "
Jul 17 15:30:23.735816 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.735791    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "08f07a98-c706-4706-9dad-fcc894fe40c3" (UID: "08f07a98-c706-4706-9dad-fcc894fe40c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:30:23.737697 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.737660    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "08f07a98-c706-4706-9dad-fcc894fe40c3" (UID: "08f07a98-c706-4706-9dad-fcc894fe40c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:30:23.737697 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.737670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p" (OuterVolumeSpecName: "kube-api-access-nk89p") pod "08f07a98-c706-4706-9dad-fcc894fe40c3" (UID: "08f07a98-c706-4706-9dad-fcc894fe40c3"). InnerVolumeSpecName "kube-api-access-nk89p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:30:23.836824 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.836788    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk89p\" (UniqueName: \"kubernetes.io/projected/08f07a98-c706-4706-9dad-fcc894fe40c3-kube-api-access-nk89p\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:30:23.836824 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.836818    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f07a98-c706-4706-9dad-fcc894fe40c3-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:30:23.836824 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.836828    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08f07a98-c706-4706-9dad-fcc894fe40c3-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:30:23.982366 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.982337    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:23.984155 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:23.984134    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-afffcc69-754d8ff48f-gj7b7"]
Jul 17 15:30:25.950880 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:25.950841    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" path="/var/lib/kubelet/pods/08f07a98-c706-4706-9dad-fcc894fe40c3/volumes"
Jul 17 15:30:26.192887 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:30:26.192846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:31:02.620515 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620479    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620858    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerName="maas-api"
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620869    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerName="maas-api"
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620882    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72cba91e-74f3-4b93-b009-cd5755be1353" containerName="revoke-keys"
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620887    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="72cba91e-74f3-4b93-b009-cd5755be1353" containerName="revoke-keys"
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620940    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f07a98-c706-4706-9dad-fcc894fe40c3" containerName="maas-api"
Jul 17 15:31:02.621112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.620948    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="72cba91e-74f3-4b93-b009-cd5755be1353" containerName="revoke-keys"
Jul 17 15:31:02.624161 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.624142    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.626997 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.626970    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:31:02.627188 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.627051    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:31:02.627265 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.627218    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a502a35a-serving-cert\""
Jul 17 15:31:02.628060 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.628039    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:31:02.636597 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.636571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:02.683736 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.683703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.683736 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.683740    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-994km\" (UniqueName: \"kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.683952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.683767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.784719 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.784681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.784719 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.784719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-994km\" (UniqueName: \"kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.784953 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.784747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.784953 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:31:02.784846    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-serving-cert: secret "maas-api-e2e-mt-a502a35a-serving-cert" not found
Jul 17 15:31:02.784953 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:31:02.784925    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls podName:7976734a-868a-406c-a573-a617d843a6bb nodeName:}" failed. No retries permitted until 2026-07-17 15:31:03.28490435 +0000 UTC m=+2473.851935463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls") pod "maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" (UID: "7976734a-868a-406c-a573-a617d843a6bb") : secret "maas-api-e2e-mt-a502a35a-serving-cert" not found
Jul 17 15:31:02.785420 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.785398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:02.794135 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:02.794109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-994km\" (UniqueName: \"kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:03.288486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.288450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:03.290861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.290839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") pod \"maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:03.536121 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.536085    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:03.679186 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.679158    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:03.681027 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:31:03.681005    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7976734a_868a_406c_a573_a617d843a6bb.slice/crio-c19fba6881bcc532dfa166d56eb64e42fb72b7d709361dae8ab901173b53af23 WatchSource:0}: Error finding container c19fba6881bcc532dfa166d56eb64e42fb72b7d709361dae8ab901173b53af23: Status 404 returned error can't find the container with id c19fba6881bcc532dfa166d56eb64e42fb72b7d709361dae8ab901173b53af23
Jul 17 15:31:03.682284 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.682266    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:31:03.869387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:03.869349    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" event={"ID":"7976734a-868a-406c-a573-a617d843a6bb","Type":"ContainerStarted","Data":"c19fba6881bcc532dfa166d56eb64e42fb72b7d709361dae8ab901173b53af23"}
Jul 17 15:31:04.875831 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:04.875792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" event={"ID":"7976734a-868a-406c-a573-a617d843a6bb","Type":"ContainerStarted","Data":"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"}
Jul 17 15:31:04.876260 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:04.875866    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:04.928173 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:04.928120    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" podStartSLOduration=2.506979049 podStartE2EDuration="2.928097871s" totalImagesPullingTime="421.118822ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:31:04.926597482 +0000 UTC m=+2475.493628628" watchObservedRunningTime="2026-07-17 15:31:04.928097871 +0000 UTC m=+2475.495129004"
Jul 17 15:31:05.313835 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:05.313799    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:31:10.889337 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:10.889305    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:16.720875 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:16.720832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:31:34.490057 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:34.490018    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:31:40.081771 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.081734    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"]
Jul 17 15:31:40.085233 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.085218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:40.087545 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.087519    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:31:40.094733 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.094707    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"]
Jul 17 15:31:40.117981 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.117955    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a502a35a-cd898\" (UID: \"50002337-2571-4a85-aa51-c9f52263d98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:40.218903 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.218867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a502a35a-cd898\" (UID: \"50002337-2571-4a85-aa51-c9f52263d98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:40.219550 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.219529    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a502a35a-cd898\" (UID: \"50002337-2571-4a85-aa51-c9f52263d98e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:40.396441 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.396348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:40.532797 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:40.532768    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"]
Jul 17 15:31:40.536442 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:31:40.536408    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50002337_2571_4a85_aa51_c9f52263d98e.slice/crio-dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5 WatchSource:0}: Error finding container dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5: Status 404 returned error can't find the container with id dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5
Jul 17 15:31:41.056710 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:41.056622    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898" event={"ID":"50002337-2571-4a85-aa51-c9f52263d98e","Type":"ContainerStarted","Data":"9008fffa66a5312ca03148c3b6bc2c3fae6cf87c9397ee88833c5def0828589b"}
Jul 17 15:31:41.056710 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:41.056677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898" event={"ID":"50002337-2571-4a85-aa51-c9f52263d98e","Type":"ContainerStarted","Data":"dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5"}
Jul 17 15:31:41.086786 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:41.086734    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898" podStartSLOduration=1.086717371 podStartE2EDuration="1.086717371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:31:41.084795304 +0000 UTC m=+2511.651826435" watchObservedRunningTime="2026-07-17 15:31:41.086717371 +0000 UTC m=+2511.653748503"
Jul 17 15:31:42.063422 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:42.063387    2564 generic.go:356] "Generic (PLEG): container finished" podID="50002337-2571-4a85-aa51-c9f52263d98e" containerID="9008fffa66a5312ca03148c3b6bc2c3fae6cf87c9397ee88833c5def0828589b" exitCode=0
Jul 17 15:31:42.063629 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:42.063447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898" event={"ID":"50002337-2571-4a85-aa51-c9f52263d98e","Type":"ContainerDied","Data":"9008fffa66a5312ca03148c3b6bc2c3fae6cf87c9397ee88833c5def0828589b"}
Jul 17 15:31:43.207660 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:43.207639    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:43.248596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:43.248560    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca\") pod \"50002337-2571-4a85-aa51-c9f52263d98e\" (UID: \"50002337-2571-4a85-aa51-c9f52263d98e\") "
Jul 17 15:31:43.248892 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:43.248864    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50002337-2571-4a85-aa51-c9f52263d98e" (UID: "50002337-2571-4a85-aa51-c9f52263d98e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:31:43.349817 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:43.349755    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50002337-2571-4a85-aa51-c9f52263d98e-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:31:44.074522 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:44.074488    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898"
Jul 17 15:31:44.074678 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:44.074518    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a502a35a-cd898" event={"ID":"50002337-2571-4a85-aa51-c9f52263d98e","Type":"ContainerDied","Data":"dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5"}
Jul 17 15:31:44.074678 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:44.074561    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbcc0e86049a345bb3de2e64711fc964830e211f6f985ace459cc0c0c5c9cea5"
Jul 17 15:31:51.229573 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.229413    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:51.230332 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.229756    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" podUID="7976734a-868a-406c-a573-a617d843a6bb" containerName="maas-api" containerID="cri-o://9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f" gracePeriod=30
Jul 17 15:31:51.478678 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.478652    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:51.617911 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.617878    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca\") pod \"7976734a-868a-406c-a573-a617d843a6bb\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") "
Jul 17 15:31:51.618107 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.617921    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") pod \"7976734a-868a-406c-a573-a617d843a6bb\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") "
Jul 17 15:31:51.618107 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.617987    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-994km\" (UniqueName: \"kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km\") pod \"7976734a-868a-406c-a573-a617d843a6bb\" (UID: \"7976734a-868a-406c-a573-a617d843a6bb\") "
Jul 17 15:31:51.618311 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.618277    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7976734a-868a-406c-a573-a617d843a6bb" (UID: "7976734a-868a-406c-a573-a617d843a6bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:31:51.620143 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.620117    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7976734a-868a-406c-a573-a617d843a6bb" (UID: "7976734a-868a-406c-a573-a617d843a6bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:31:51.620252 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.620153    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km" (OuterVolumeSpecName: "kube-api-access-994km") pod "7976734a-868a-406c-a573-a617d843a6bb" (UID: "7976734a-868a-406c-a573-a617d843a6bb"). InnerVolumeSpecName "kube-api-access-994km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:31:51.719057 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.719023    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7976734a-868a-406c-a573-a617d843a6bb-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:31:51.719057 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.719051    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7976734a-868a-406c-a573-a617d843a6bb-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:31:51.719243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:51.719083    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-994km\" (UniqueName: \"kubernetes.io/projected/7976734a-868a-406c-a573-a617d843a6bb-kube-api-access-994km\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:31:52.113639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.113605    2564 generic.go:356] "Generic (PLEG): container finished" podID="7976734a-868a-406c-a573-a617d843a6bb" containerID="9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f" exitCode=0
Jul 17 15:31:52.113639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.113638    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"
Jul 17 15:31:52.113843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.113675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" event={"ID":"7976734a-868a-406c-a573-a617d843a6bb","Type":"ContainerDied","Data":"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"}
Jul 17 15:31:52.113843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.113730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9" event={"ID":"7976734a-868a-406c-a573-a617d843a6bb","Type":"ContainerDied","Data":"c19fba6881bcc532dfa166d56eb64e42fb72b7d709361dae8ab901173b53af23"}
Jul 17 15:31:52.113843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.113760    2564 scope.go:117] "RemoveContainer" containerID="9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"
Jul 17 15:31:52.124216 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.124195    2564 scope.go:117] "RemoveContainer" containerID="9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"
Jul 17 15:31:52.124487 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:31:52.124464    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f\": container with ID starting with 9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f not found: ID does not exist" containerID="9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"
Jul 17 15:31:52.124544 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.124499    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f"} err="failed to get container status \"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f\": rpc error: code = NotFound desc = could not find container \"9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f\": container with ID starting with 9ab9ea904adb1adc6c93bdd5c68c22f4279846a6ed7ffd620ca959dce4eaa22f not found: ID does not exist"
Jul 17 15:31:52.133872 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.133851    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:52.138592 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:52.138568    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a502a35a-658bff7fff-4cqt9"]
Jul 17 15:31:53.950079 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:53.950038    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7976734a-868a-406c-a573-a617d843a6bb" path="/var/lib/kubelet/pods/7976734a-868a-406c-a573-a617d843a6bb/volumes"
Jul 17 15:31:54.983747 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:31:54.983711    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:32:41.792125 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:32:41.792083    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:32:56.589117 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:32:56.589084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:33:38.840339 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840257    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:33:38.840784 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840669    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50002337-2571-4a85-aa51-c9f52263d98e" containerName="revoke-keys"
Jul 17 15:33:38.840784 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840686    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="50002337-2571-4a85-aa51-c9f52263d98e" containerName="revoke-keys"
Jul 17 15:33:38.840784 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840711    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976734a-868a-406c-a573-a617d843a6bb" containerName="maas-api"
Jul 17 15:33:38.840784 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840716    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976734a-868a-406c-a573-a617d843a6bb" containerName="maas-api"
Jul 17 15:33:38.840784 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840778    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="50002337-2571-4a85-aa51-c9f52263d98e" containerName="revoke-keys"
Jul 17 15:33:38.840947 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.840789    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976734a-868a-406c-a573-a617d843a6bb" containerName="maas-api"
Jul 17 15:33:38.843992 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.843963    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:38.848194 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.848170    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4120cb-serving-cert\""
Jul 17 15:33:38.872260 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.872231    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:33:38.878605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.878585    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:33:38.882263 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.882247    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:33:38.886699 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.886683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:33:38.904996 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.904969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4gcg\" (UniqueName: \"kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:38.905145 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.905025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:38.905217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:38.905193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.006410 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.006372    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.006591 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.006430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4gcg\" (UniqueName: \"kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.006591 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.006478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.006732 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:33:39.006602    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-serving-cert: secret "maas-api-e2e-inf-a-4120cb-serving-cert" not found
Jul 17 15:33:39.006732 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:33:39.006698    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls podName:e5132323-92ea-4c3b-935b-843d87d5ef11 nodeName:}" failed. No retries permitted until 2026-07-17 15:33:39.506676431 +0000 UTC m=+2630.073707553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls") pod "maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" (UID: "e5132323-92ea-4c3b-935b-843d87d5ef11") : secret "maas-api-e2e-inf-a-4120cb-serving-cert" not found
Jul 17 15:33:39.007188 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.007165    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.037511 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.037478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4gcg\" (UniqueName: \"kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.510825 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.510781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.513176 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.513155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") pod \"maas-api-e2e-inf-a-4120cb-599b687d99-9btc6\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.754258 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.754221    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:39.962970 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:39.962935    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:33:39.971553 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:33:39.971525    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5132323_92ea_4c3b_935b_843d87d5ef11.slice/crio-6e97751dea43d1f291d1c276bdbdd013bb15f9682435bc3cac531a2dad6abff7 WatchSource:0}: Error finding container 6e97751dea43d1f291d1c276bdbdd013bb15f9682435bc3cac531a2dad6abff7: Status 404 returned error can't find the container with id 6e97751dea43d1f291d1c276bdbdd013bb15f9682435bc3cac531a2dad6abff7
Jul 17 15:33:40.659340 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:40.659314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" event={"ID":"e5132323-92ea-4c3b-935b-843d87d5ef11","Type":"ContainerStarted","Data":"6e97751dea43d1f291d1c276bdbdd013bb15f9682435bc3cac531a2dad6abff7"}
Jul 17 15:33:41.666507 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:41.666472    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" event={"ID":"e5132323-92ea-4c3b-935b-843d87d5ef11","Type":"ContainerStarted","Data":"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"}
Jul 17 15:33:41.666996 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:41.666540    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:41.692072 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:41.692019    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" podStartSLOduration=3.017419646 podStartE2EDuration="3.692002966s" totalImagesPullingTime="674.58332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:33:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:33:41.690518363 +0000 UTC m=+2632.257549495" watchObservedRunningTime="2026-07-17 15:33:41.692002966 +0000 UTC m=+2632.259034097"
Jul 17 15:33:46.081299 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.081268    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:33:46.604508 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.604471    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:33:46.608726 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.608702    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.612776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.612753    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c10dc4-serving-cert\""
Jul 17 15:33:46.620278 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.620253    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:33:46.677481 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.677446    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c87c\" (UniqueName: \"kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.677653 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.677507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.677653 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.677603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.778253 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.778222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c87c\" (UniqueName: \"kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.778435 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.778282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.778435 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.778332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.778958 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.778936    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.780928 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.780906    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.789853 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.789826    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c87c\" (UniqueName: \"kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c\") pod \"maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:46.920589 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:46.920504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:47.077951 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.077914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:33:47.089275 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:33:47.089244    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cacd9fa_d01c_4493_a5e0_0a6c26ce27ac.slice/crio-901cbbd437413faba179d2590fb9494565f8733bf9b0f2f892f475887c412640 WatchSource:0}: Error finding container 901cbbd437413faba179d2590fb9494565f8733bf9b0f2f892f475887c412640: Status 404 returned error can't find the container with id 901cbbd437413faba179d2590fb9494565f8733bf9b0f2f892f475887c412640
Jul 17 15:33:47.678039 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.678008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:33:47.714210 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.714108    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" event={"ID":"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac","Type":"ContainerStarted","Data":"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"}
Jul 17 15:33:47.714376 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.714215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" event={"ID":"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac","Type":"ContainerStarted","Data":"901cbbd437413faba179d2590fb9494565f8733bf9b0f2f892f475887c412640"}
Jul 17 15:33:47.714376 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.714252    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:47.739970 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:47.739918    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" podStartSLOduration=1.370219142 podStartE2EDuration="1.739898891s" totalImagesPullingTime="369.679749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:33:47.738629855 +0000 UTC m=+2638.305661007" watchObservedRunningTime="2026-07-17 15:33:47.739898891 +0000 UTC m=+2638.306930028"
Jul 17 15:33:53.727123 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:53.727092    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:33:56.158022 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.157985    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:33:56.162623 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.162605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.165525 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.165501    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4120cb\"/\"kube-root-ca.crt\""
Jul 17 15:33:56.165675 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.165556    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4120cb\"/\"openshift-service-ca.crt\""
Jul 17 15:33:56.165732 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.165684    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4120cb\"/\"default-dockercfg-tx4jk\""
Jul 17 15:33:56.166481 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.166466    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4120cb\"/\"test-model-4120cb-kserve-self-signed-certs\""
Jul 17 15:33:56.178409 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.178390    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:33:56.260918 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.260881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.260918 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.260913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.261131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.260962    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.261131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.260987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.261131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.261014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.261131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.261099    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtm6l\" (UniqueName: \"kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362440 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362440 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362445    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362644 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362644 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtm6l\" (UniqueName: \"kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362644 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362644 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.362952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362930    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.363025 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.363025 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.362996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.364788 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.364768    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.364964 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.364946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.375201 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.375175    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtm6l\" (UniqueName: \"kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l\") pod \"test-model-4120cb-kserve-8db79594f-k6tp6\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") " pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.472323 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.472252    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:33:56.611489 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.611467    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:33:56.613785 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:33:56.613757    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4de5e704_fa1f_4fee_a667_06f8e57c34d5.slice/crio-1c1d8cb581315441b3c1dc867eb76cc39eea238deafa61087b443825b0badaa9 WatchSource:0}: Error finding container 1c1d8cb581315441b3c1dc867eb76cc39eea238deafa61087b443825b0badaa9: Status 404 returned error can't find the container with id 1c1d8cb581315441b3c1dc867eb76cc39eea238deafa61087b443825b0badaa9
Jul 17 15:33:56.764030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.763941    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" event={"ID":"4de5e704-fa1f-4fee-a667-06f8e57c34d5","Type":"ContainerStarted","Data":"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"}
Jul 17 15:33:56.764030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.763991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" event={"ID":"4de5e704-fa1f-4fee-a667-06f8e57c34d5","Type":"ContainerStarted","Data":"1c1d8cb581315441b3c1dc867eb76cc39eea238deafa61087b443825b0badaa9"}
Jul 17 15:33:56.799341 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:56.799295    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" podStartSLOduration=0.799277273 podStartE2EDuration="799.277273ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:33:56.798321787 +0000 UTC m=+2647.365352931" watchObservedRunningTime="2026-07-17 15:33:56.799277273 +0000 UTC m=+2647.366308404"
Jul 17 15:33:59.131985 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:33:59.131940    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:34:06.472660 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:06.472624    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:34:06.473089 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:06.472698    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:34:06.485082 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:06.485041    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:34:06.824576 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:06.824550    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:34:15.128753 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:15.128714    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:34:29.591450 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:29.591410    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:34:43.784385 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:43.784305    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:34:49.449426 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:49.449392    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:34:50.082081 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.082033    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:34:50.090526 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.090500    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:34:50.100538 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.100514    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:34:50.105406 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.105389    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:34:50.492728 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.492651    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"]
Jul 17 15:34:50.496436 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.496417    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:50.499250 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.499229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:34:50.505850 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.505817    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"]
Jul 17 15:34:50.632782 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.632747    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46\" (UID: \"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:50.733315 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.733285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46\" (UID: \"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:50.733875 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.733855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46\" (UID: \"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:50.809502 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.809431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:50.952564 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:50.952536    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"]
Jul 17 15:34:50.953722 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:34:50.953697    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod816a1ae9_f7d1_44a0_bfdb_ac6789c54e5a.slice/crio-77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee WatchSource:0}: Error finding container 77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee: Status 404 returned error can't find the container with id 77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee
Jul 17 15:34:51.043416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:51.043373    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46" event={"ID":"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a","Type":"ContainerStarted","Data":"77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee"}
Jul 17 15:34:52.050024 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:52.049935    2564 generic.go:356] "Generic (PLEG): container finished" podID="816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a" containerID="c7a68e9cce4058b276ef63dde1f3ab59588075e8c78466e2126964b9ab1e0537" exitCode=0
Jul 17 15:34:52.050024 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:52.050007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46" event={"ID":"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a","Type":"ContainerDied","Data":"c7a68e9cce4058b276ef63dde1f3ab59588075e8c78466e2126964b9ab1e0537"}
Jul 17 15:34:53.193316 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:53.193294    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:53.255697 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:53.255668    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca\") pod \"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a\" (UID: \"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a\") "
Jul 17 15:34:53.256021 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:53.256001    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a" (UID: "816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:34:53.357256 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:53.357224    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:34:54.060946 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:54.060918    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46"
Jul 17 15:34:54.061137 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:54.060918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46" event={"ID":"816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a","Type":"ContainerDied","Data":"77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee"}
Jul 17 15:34:54.061137 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:54.061043    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77b31a4a43bb4eec77d6a6b5147499b0d0cdf8a63a637f6a30cf7a0780e35cee"
Jul 17 15:34:56.897006 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:34:56.896971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:35:01.053032 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.052995    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:35:01.053795 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.053760    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" podUID="e5132323-92ea-4c3b-935b-843d87d5ef11" containerName="maas-api" containerID="cri-o://d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179" gracePeriod=30
Jul 17 15:35:01.309445 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.309382    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:35:01.428572 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.428534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4gcg\" (UniqueName: \"kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg\") pod \"e5132323-92ea-4c3b-935b-843d87d5ef11\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") "
Jul 17 15:35:01.428754 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.428614    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca\") pod \"e5132323-92ea-4c3b-935b-843d87d5ef11\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") "
Jul 17 15:35:01.428754 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.428662    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") pod \"e5132323-92ea-4c3b-935b-843d87d5ef11\" (UID: \"e5132323-92ea-4c3b-935b-843d87d5ef11\") "
Jul 17 15:35:01.429010 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.428984    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e5132323-92ea-4c3b-935b-843d87d5ef11" (UID: "e5132323-92ea-4c3b-935b-843d87d5ef11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:35:01.430757 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.430719    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg" (OuterVolumeSpecName: "kube-api-access-d4gcg") pod "e5132323-92ea-4c3b-935b-843d87d5ef11" (UID: "e5132323-92ea-4c3b-935b-843d87d5ef11"). InnerVolumeSpecName "kube-api-access-d4gcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:35:01.430894 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.430769    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e5132323-92ea-4c3b-935b-843d87d5ef11" (UID: "e5132323-92ea-4c3b-935b-843d87d5ef11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:35:01.529523 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.529479    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5132323-92ea-4c3b-935b-843d87d5ef11-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:01.529523 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.529517    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5132323-92ea-4c3b-935b-843d87d5ef11-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:01.529523 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:01.529531    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4gcg\" (UniqueName: \"kubernetes.io/projected/e5132323-92ea-4c3b-935b-843d87d5ef11-kube-api-access-d4gcg\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:02.103699 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.103665    2564 generic.go:356] "Generic (PLEG): container finished" podID="e5132323-92ea-4c3b-935b-843d87d5ef11" containerID="d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179" exitCode=0
Jul 17 15:35:02.104217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.103716    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"
Jul 17 15:35:02.104217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.103746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" event={"ID":"e5132323-92ea-4c3b-935b-843d87d5ef11","Type":"ContainerDied","Data":"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"}
Jul 17 15:35:02.104217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.103800    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6" event={"ID":"e5132323-92ea-4c3b-935b-843d87d5ef11","Type":"ContainerDied","Data":"6e97751dea43d1f291d1c276bdbdd013bb15f9682435bc3cac531a2dad6abff7"}
Jul 17 15:35:02.104217 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.103823    2564 scope.go:117] "RemoveContainer" containerID="d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"
Jul 17 15:35:02.112688 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.112670    2564 scope.go:117] "RemoveContainer" containerID="d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"
Jul 17 15:35:02.112986 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:02.112968    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179\": container with ID starting with d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179 not found: ID does not exist" containerID="d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"
Jul 17 15:35:02.113041 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.112996    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179"} err="failed to get container status \"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179\": rpc error: code = NotFound desc = could not find container \"d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179\": container with ID starting with d7567c4d0b7c88f1dd78032aa6ea528f19bc2ed3eac03f695a30edc01ad0a179 not found: ID does not exist"
Jul 17 15:35:02.122757 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.122730    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:35:02.127570 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:02.127530    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4120cb-599b687d99-9btc6"]
Jul 17 15:35:03.950871 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:03.950836    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5132323-92ea-4c3b-935b-843d87d5ef11" path="/var/lib/kubelet/pods/e5132323-92ea-4c3b-935b-843d87d5ef11/volumes"
Jul 17 15:35:04.469743 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.469700    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" podUID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" containerName="main" containerID="cri-o://0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91" gracePeriod=45
Jul 17 15:35:04.723389 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.723323    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:35:04.858233 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858202    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858314    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858337    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858354    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtm6l\" (UniqueName: \"kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858374    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858406    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir\") pod \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\" (UID: \"4de5e704-fa1f-4fee-a667-06f8e57c34d5\") "
Jul 17 15:35:04.858692 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home" (OuterVolumeSpecName: "home") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:35:04.858692 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858559    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:35:04.858692 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858667    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-model-cache\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:04.858692 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858684    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-home\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:04.858836 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.858723    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:35:04.860533 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.860502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm" (OuterVolumeSpecName: "dshm") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:35:04.860625 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.860559    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:35:04.860625 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.860605    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l" (OuterVolumeSpecName: "kube-api-access-xtm6l") pod "4de5e704-fa1f-4fee-a667-06f8e57c34d5" (UID: "4de5e704-fa1f-4fee-a667-06f8e57c34d5"). InnerVolumeSpecName "kube-api-access-xtm6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:35:04.959950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.959912    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tmp-dir\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:04.959950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.959947    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4de5e704-fa1f-4fee-a667-06f8e57c34d5-dshm\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:04.959950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.959956    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtm6l\" (UniqueName: \"kubernetes.io/projected/4de5e704-fa1f-4fee-a667-06f8e57c34d5-kube-api-access-xtm6l\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:04.959950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:04.959966    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4de5e704-fa1f-4fee-a667-06f8e57c34d5-tls-certs\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:05.122229 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.122197    2564 generic.go:356] "Generic (PLEG): container finished" podID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" containerID="0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91" exitCode=0
Jul 17 15:35:05.122417 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.122240    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"
Jul 17 15:35:05.122417 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.122252    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" event={"ID":"4de5e704-fa1f-4fee-a667-06f8e57c34d5","Type":"ContainerDied","Data":"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"}
Jul 17 15:35:05.122417 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.122285    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6" event={"ID":"4de5e704-fa1f-4fee-a667-06f8e57c34d5","Type":"ContainerDied","Data":"1c1d8cb581315441b3c1dc867eb76cc39eea238deafa61087b443825b0badaa9"}
Jul 17 15:35:05.122417 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.122302    2564 scope.go:117] "RemoveContainer" containerID="0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"
Jul 17 15:35:05.131587 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.131556    2564 scope.go:117] "RemoveContainer" containerID="0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"
Jul 17 15:35:05.131825 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:05.131808    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91\": container with ID starting with 0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91 not found: ID does not exist" containerID="0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"
Jul 17 15:35:05.131891 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.131831    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91"} err="failed to get container status \"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91\": rpc error: code = NotFound desc = could not find container \"0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91\": container with ID starting with 0db1e75543489e3fc23d8ebeac31e8db67466db43983695e80cd04ce80e9cc91 not found: ID does not exist"
Jul 17 15:35:05.147470 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.147446    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:35:05.152182 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.152162    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4120cb/test-model-4120cb-kserve-8db79594f-k6tp6"]
Jul 17 15:35:05.950174 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:05.950138    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" path="/var/lib/kubelet/pods/4de5e704-fa1f-4fee-a667-06f8e57c34d5/volumes"
Jul 17 15:35:10.991495 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:10.991451    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:35:16.725636 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.725603    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"]
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.725972    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a" containerName="revoke-keys"
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.725984    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a" containerName="revoke-keys"
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.725997    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" containerName="main"
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726002    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" containerName="main"
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726010    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5132323-92ea-4c3b-935b-843d87d5ef11" containerName="maas-api"
Jul 17 15:35:16.726030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726016    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5132323-92ea-4c3b-935b-843d87d5ef11" containerName="maas-api"
Jul 17 15:35:16.726269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726103    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a" containerName="revoke-keys"
Jul 17 15:35:16.726269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726111    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de5e704-fa1f-4fee-a667-06f8e57c34d5" containerName="main"
Jul 17 15:35:16.726269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.726117    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5132323-92ea-4c3b-935b-843d87d5ef11" containerName="maas-api"
Jul 17 15:35:16.730635 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.730617    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:16.735221 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.735203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:35:16.748416 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.748392    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"]
Jul 17 15:35:16.767658 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.767632    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c\" (UID: \"0855e14a-a49f-4ae7-bb0e-c8759805e840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:16.868165 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.868129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c\" (UID: \"0855e14a-a49f-4ae7-bb0e-c8759805e840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:16.868712 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:16.868693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c\" (UID: \"0855e14a-a49f-4ae7-bb0e-c8759805e840\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:17.040979 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:17.040904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:17.191948 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:17.191926    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"]
Jul 17 15:35:17.193948 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:35:17.193923    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0855e14a_a49f_4ae7_bb0e_c8759805e840.slice/crio-f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a WatchSource:0}: Error finding container f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a: Status 404 returned error can't find the container with id f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a
Jul 17 15:35:18.190206 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:18.190109    2564 generic.go:356] "Generic (PLEG): container finished" podID="0855e14a-a49f-4ae7-bb0e-c8759805e840" containerID="e9f716ca44d77ed279e597eb7ed48353185cfd33edbd3e85a2a8ee5c21d01b64" exitCode=0
Jul 17 15:35:18.190206 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:18.190149    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c" event={"ID":"0855e14a-a49f-4ae7-bb0e-c8759805e840","Type":"ContainerDied","Data":"e9f716ca44d77ed279e597eb7ed48353185cfd33edbd3e85a2a8ee5c21d01b64"}
Jul 17 15:35:18.190653 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:18.190201    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c" event={"ID":"0855e14a-a49f-4ae7-bb0e-c8759805e840","Type":"ContainerStarted","Data":"f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a"}
Jul 17 15:35:19.333467 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:19.333443    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:19.386937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:19.386914    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca\") pod \"0855e14a-a49f-4ae7-bb0e-c8759805e840\" (UID: \"0855e14a-a49f-4ae7-bb0e-c8759805e840\") "
Jul 17 15:35:19.387232 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:19.387212    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0855e14a-a49f-4ae7-bb0e-c8759805e840" (UID: "0855e14a-a49f-4ae7-bb0e-c8759805e840"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:35:19.387325 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:19.387301    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0855e14a-a49f-4ae7-bb0e-c8759805e840-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:20.200812 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:20.200784    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c"
Jul 17 15:35:20.200977 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:20.200824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c" event={"ID":"0855e14a-a49f-4ae7-bb0e-c8759805e840","Type":"ContainerDied","Data":"f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a"}
Jul 17 15:35:20.200977 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:20.200873    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f93c5c65e25d07f6c12657c1d6bda5ca1843b66c3d20c589afbb03e47afc188a"
Jul 17 15:35:27.257595 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.257559    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:35:27.258204 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.257799    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" podUID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" containerName="maas-api" containerID="cri-o://ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a" gracePeriod=30
Jul 17 15:35:27.508129 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.508056    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:35:27.553576 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.553540    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca\") pod \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") "
Jul 17 15:35:27.553717 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.553661    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls\") pod \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") "
Jul 17 15:35:27.553756 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.553738    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c87c\" (UniqueName: \"kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c\") pod \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\" (UID: \"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac\") "
Jul 17 15:35:27.553933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.553906    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" (UID: "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:35:27.554001 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.553981    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:27.555808 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.555784    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" (UID: "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:35:27.555911 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.555784    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c" (OuterVolumeSpecName: "kube-api-access-9c87c") pod "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" (UID: "3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac"). InnerVolumeSpecName "kube-api-access-9c87c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:35:27.654676 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.654649    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:27.654676 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:27.654677    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c87c\" (UniqueName: \"kubernetes.io/projected/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac-kube-api-access-9c87c\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:35:28.245596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.245516    2564 generic.go:356] "Generic (PLEG): container finished" podID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" containerID="ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a" exitCode=0
Jul 17 15:35:28.245596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.245560    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"
Jul 17 15:35:28.245843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.245615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" event={"ID":"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac","Type":"ContainerDied","Data":"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"}
Jul 17 15:35:28.245843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.245673    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c" event={"ID":"3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac","Type":"ContainerDied","Data":"901cbbd437413faba179d2590fb9494565f8733bf9b0f2f892f475887c412640"}
Jul 17 15:35:28.245843 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.245702    2564 scope.go:117] "RemoveContainer" containerID="ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"
Jul 17 15:35:28.254648 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.254632    2564 scope.go:117] "RemoveContainer" containerID="ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"
Jul 17 15:35:28.254891 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:28.254874    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a\": container with ID starting with ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a not found: ID does not exist" containerID="ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"
Jul 17 15:35:28.254937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.254899    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a"} err="failed to get container status \"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a\": rpc error: code = NotFound desc = could not find container \"ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a\": container with ID starting with ae05ea044a07127229c5215be44871f132d392227aad48f483412a9944cc904a not found: ID does not exist"
Jul 17 15:35:28.263300 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.263279    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:35:28.267429 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:28.267406    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c10dc4-58cdc7dc8c-5dg2c"]
Jul 17 15:35:29.950503 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:29.950460    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" path="/var/lib/kubelet/pods/3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac/volumes"
Jul 17 15:35:30.584546 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:30.584515    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:35:46.333313 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333272    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:35:46.333819 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333804    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" containerName="maas-api"
Jul 17 15:35:46.333863 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" containerName="maas-api"
Jul 17 15:35:46.333899 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333863    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0855e14a-a49f-4ae7-bb0e-c8759805e840" containerName="revoke-keys"
Jul 17 15:35:46.333899 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333873    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0855e14a-a49f-4ae7-bb0e-c8759805e840" containerName="revoke-keys"
Jul 17 15:35:46.334008 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333964    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cacd9fa-d01c-4493-a5e0-0a6c26ce27ac" containerName="maas-api"
Jul 17 15:35:46.334008 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.333979    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0855e14a-a49f-4ae7-bb0e-c8759805e840" containerName="revoke-keys"
Jul 17 15:35:46.340506 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.340476    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.343847 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.343827    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:35:46.343990 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.343852    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:35:46.344398 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.344378    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c30485-serving-cert\""
Jul 17 15:35:46.344508 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.344398    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:35:46.352207 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.352181    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:35:46.437632 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.437593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.437789 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.437677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.437789 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.437712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqfdg\" (UniqueName: \"kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.539193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.539150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqfdg\" (UniqueName: \"kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.539385 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.539234    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.539385 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.539324    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.539525 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:46.539449    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-serving-cert: secret "maas-api-e2e-api-a-c30485-serving-cert" not found
Jul 17 15:35:46.539585 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:46.539564    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls podName:13559f84-7149-47ab-a696-8e2ab2542dc4 nodeName:}" failed. No retries permitted until 2026-07-17 15:35:47.039535066 +0000 UTC m=+2757.606566190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls") pod "maas-api-e2e-api-a-c30485-855d566f5-fv9rs" (UID: "13559f84-7149-47ab-a696-8e2ab2542dc4") : secret "maas-api-e2e-api-a-c30485-serving-cert" not found
Jul 17 15:35:46.539972 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.539946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:46.559329 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:46.559297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqfdg\" (UniqueName: \"kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:47.045253 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:47.045221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:47.047643 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:47.047609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") pod \"maas-api-e2e-api-a-c30485-855d566f5-fv9rs\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:47.252963 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:47.252924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:47.397011 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:47.396986    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:35:47.398726 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:35:47.398697    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13559f84_7149_47ab_a696_8e2ab2542dc4.slice/crio-6806237c0736a01dc7e4de1d22c27c6ded2c4895c456b1291fea4b3048e710ab WatchSource:0}: Error finding container 6806237c0736a01dc7e4de1d22c27c6ded2c4895c456b1291fea4b3048e710ab: Status 404 returned error can't find the container with id 6806237c0736a01dc7e4de1d22c27c6ded2c4895c456b1291fea4b3048e710ab
Jul 17 15:35:48.357487 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:48.357448    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" event={"ID":"13559f84-7149-47ab-a696-8e2ab2542dc4","Type":"ContainerStarted","Data":"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"}
Jul 17 15:35:48.357661 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:48.357502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" event={"ID":"13559f84-7149-47ab-a696-8e2ab2542dc4","Type":"ContainerStarted","Data":"6806237c0736a01dc7e4de1d22c27c6ded2c4895c456b1291fea4b3048e710ab"}
Jul 17 15:35:48.357752 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:48.357733    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:48.383827 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:48.383784    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" podStartSLOduration=2.006595136 podStartE2EDuration="2.383771726s" totalImagesPullingTime="377.17659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:35:48.381637182 +0000 UTC m=+2758.948668314" watchObservedRunningTime="2026-07-17 15:35:48.383771726 +0000 UTC m=+2758.950802857"
Jul 17 15:35:53.283846 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:53.283801    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:35:54.368892 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.368859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:35:54.382728 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.382697    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:35:54.388096 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.388060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.390933 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.390910    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-818628-serving-cert\""
Jul 17 15:35:54.401160 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.400924    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:35:54.511758 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.511724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.511970 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.511870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmqdn\" (UniqueName: \"kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.511970 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.511926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.612576 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.612539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.612775 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.612594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmqdn\" (UniqueName: \"kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.612775 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.612624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.612775 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:54.612734    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-818628-serving-cert: secret "maas-api-e2e-api-b-818628-serving-cert" not found
Jul 17 15:35:54.612945 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:35:54.612784    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls podName:b6bd94b6-1be6-4164-a069-43289040e339 nodeName:}" failed. No retries permitted until 2026-07-17 15:35:55.11276932 +0000 UTC m=+2765.679800433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls") pod "maas-api-e2e-api-b-818628-6f99796498-kb2nc" (UID: "b6bd94b6-1be6-4164-a069-43289040e339") : secret "maas-api-e2e-api-b-818628-serving-cert" not found
Jul 17 15:35:54.613281 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.613260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:54.623497 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:54.623427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmqdn\" (UniqueName: \"kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:55.118475 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:55.118432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:55.121118 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:55.121090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") pod \"maas-api-e2e-api-b-818628-6f99796498-kb2nc\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:55.302294 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:55.302254    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:55.464274 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:55.464249    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:35:55.465695 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:35:55.465667    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6bd94b6_1be6_4164_a069_43289040e339.slice/crio-7231aa7130cd6c0fd9c15376b61a8335d256c8989c8fa74448b666844a14cdfa WatchSource:0}: Error finding container 7231aa7130cd6c0fd9c15376b61a8335d256c8989c8fa74448b666844a14cdfa: Status 404 returned error can't find the container with id 7231aa7130cd6c0fd9c15376b61a8335d256c8989c8fa74448b666844a14cdfa
Jul 17 15:35:56.408138 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:56.408100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" event={"ID":"b6bd94b6-1be6-4164-a069-43289040e339","Type":"ContainerStarted","Data":"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"}
Jul 17 15:35:56.408328 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:56.408309    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:35:56.408379 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:56.408352    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" event={"ID":"b6bd94b6-1be6-4164-a069-43289040e339","Type":"ContainerStarted","Data":"7231aa7130cd6c0fd9c15376b61a8335d256c8989c8fa74448b666844a14cdfa"}
Jul 17 15:35:56.430461 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:35:56.430416    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" podStartSLOduration=2.097631257 podStartE2EDuration="2.4304007s" totalImagesPullingTime="332.769443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:35:56.429139299 +0000 UTC m=+2766.996170452" watchObservedRunningTime="2026-07-17 15:35:56.4304007 +0000 UTC m=+2766.997431832"
Jul 17 15:36:02.419151 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:02.419121    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:36:06.582630 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.582591    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"]
Jul 17 15:36:06.586332 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.586316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:06.588646 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.588628    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:36:06.594749 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.594727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"]
Jul 17 15:36:06.718263 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.718229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c30485-6pctp\" (UID: \"55cbaac1-7bb4-4fd5-949b-29cb385677c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:06.819639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.819610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c30485-6pctp\" (UID: \"55cbaac1-7bb4-4fd5-949b-29cb385677c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:06.820382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.820360    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c30485-6pctp\" (UID: \"55cbaac1-7bb4-4fd5-949b-29cb385677c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:06.898509 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:06.898448    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:07.037689 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:07.037657    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"]
Jul 17 15:36:07.039274 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:36:07.039248    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cbaac1_7bb4_4fd5_949b_29cb385677c7.slice/crio-f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555 WatchSource:0}: Error finding container f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555: Status 404 returned error can't find the container with id f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555
Jul 17 15:36:07.467231 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:07.467200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp" event={"ID":"55cbaac1-7bb4-4fd5-949b-29cb385677c7","Type":"ContainerStarted","Data":"3b72b92b4a9f4c286cd712fb6271106771161fa17406b6bdfd3ff23c7adc91b0"}
Jul 17 15:36:07.467410 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:07.467251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp" event={"ID":"55cbaac1-7bb4-4fd5-949b-29cb385677c7","Type":"ContainerStarted","Data":"f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555"}
Jul 17 15:36:07.485723 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:07.485638    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp" podStartSLOduration=1.485623092 podStartE2EDuration="1.485623092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:36:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:36:07.483338757 +0000 UTC m=+2778.050369915" watchObservedRunningTime="2026-07-17 15:36:07.485623092 +0000 UTC m=+2778.052654224"
Jul 17 15:36:08.473289 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:08.473260    2564 generic.go:356] "Generic (PLEG): container finished" podID="55cbaac1-7bb4-4fd5-949b-29cb385677c7" containerID="3b72b92b4a9f4c286cd712fb6271106771161fa17406b6bdfd3ff23c7adc91b0" exitCode=0
Jul 17 15:36:08.473677 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:08.473314    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp" event={"ID":"55cbaac1-7bb4-4fd5-949b-29cb385677c7","Type":"ContainerDied","Data":"3b72b92b4a9f4c286cd712fb6271106771161fa17406b6bdfd3ff23c7adc91b0"}
Jul 17 15:36:09.382973 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:09.382939    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:36:09.620811 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:09.620783    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:09.644756 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:09.644685    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca\") pod \"55cbaac1-7bb4-4fd5-949b-29cb385677c7\" (UID: \"55cbaac1-7bb4-4fd5-949b-29cb385677c7\") "
Jul 17 15:36:09.645044 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:09.645023    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55cbaac1-7bb4-4fd5-949b-29cb385677c7" (UID: "55cbaac1-7bb4-4fd5-949b-29cb385677c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:36:09.745923 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:09.745890    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cbaac1-7bb4-4fd5-949b-29cb385677c7-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:10.485767 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:10.485735    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp"
Jul 17 15:36:10.485767 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:10.485748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c30485-6pctp" event={"ID":"55cbaac1-7bb4-4fd5-949b-29cb385677c7","Type":"ContainerDied","Data":"f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555"}
Jul 17 15:36:10.485971 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:10.485796    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f059b4810bcb97cc6e50a4e7489a7591b67c2387f937fe5e5d1a1175f9bfc555"
Jul 17 15:36:16.681353 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:16.681262    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:36:16.681785 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:16.681534    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" podUID="13559f84-7149-47ab-a696-8e2ab2542dc4" containerName="maas-api" containerID="cri-o://6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1" gracePeriod=30
Jul 17 15:36:16.940274 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:16.940217    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:36:17.010341 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.010315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") pod \"13559f84-7149-47ab-a696-8e2ab2542dc4\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") "
Jul 17 15:36:17.010511 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.010356    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqfdg\" (UniqueName: \"kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg\") pod \"13559f84-7149-47ab-a696-8e2ab2542dc4\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") "
Jul 17 15:36:17.010511 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.010432    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca\") pod \"13559f84-7149-47ab-a696-8e2ab2542dc4\" (UID: \"13559f84-7149-47ab-a696-8e2ab2542dc4\") "
Jul 17 15:36:17.010830 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.010802    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13559f84-7149-47ab-a696-8e2ab2542dc4" (UID: "13559f84-7149-47ab-a696-8e2ab2542dc4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:36:17.012350 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.012323    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg" (OuterVolumeSpecName: "kube-api-access-cqfdg") pod "13559f84-7149-47ab-a696-8e2ab2542dc4" (UID: "13559f84-7149-47ab-a696-8e2ab2542dc4"). InnerVolumeSpecName "kube-api-access-cqfdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:36:17.012350 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.012326    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13559f84-7149-47ab-a696-8e2ab2542dc4" (UID: "13559f84-7149-47ab-a696-8e2ab2542dc4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:36:17.111535 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.111502    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13559f84-7149-47ab-a696-8e2ab2542dc4-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:17.111535 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.111530    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13559f84-7149-47ab-a696-8e2ab2542dc4-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:17.111535 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.111540    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqfdg\" (UniqueName: \"kubernetes.io/projected/13559f84-7149-47ab-a696-8e2ab2542dc4-kube-api-access-cqfdg\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:17.522188 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.522150    2564 generic.go:356] "Generic (PLEG): container finished" podID="13559f84-7149-47ab-a696-8e2ab2542dc4" containerID="6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1" exitCode=0
Jul 17 15:36:17.522386 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.522200    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"
Jul 17 15:36:17.522386 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.522235    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" event={"ID":"13559f84-7149-47ab-a696-8e2ab2542dc4","Type":"ContainerDied","Data":"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"}
Jul 17 15:36:17.522386 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.522300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs" event={"ID":"13559f84-7149-47ab-a696-8e2ab2542dc4","Type":"ContainerDied","Data":"6806237c0736a01dc7e4de1d22c27c6ded2c4895c456b1291fea4b3048e710ab"}
Jul 17 15:36:17.522386 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.522330    2564 scope.go:117] "RemoveContainer" containerID="6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"
Jul 17 15:36:17.532157 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.532139    2564 scope.go:117] "RemoveContainer" containerID="6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"
Jul 17 15:36:17.532389 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:36:17.532367    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1\": container with ID starting with 6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1 not found: ID does not exist" containerID="6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"
Jul 17 15:36:17.532478 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.532397    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1"} err="failed to get container status \"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1\": rpc error: code = NotFound desc = could not find container \"6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1\": container with ID starting with 6478a07c7f402b7ecf561144ca20fa2ac50def516fb30c0bdcf4f8d13586f5a1 not found: ID does not exist"
Jul 17 15:36:17.545260 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.545234    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:36:17.551902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.551876    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c30485-855d566f5-fv9rs"]
Jul 17 15:36:17.950315 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:17.950282    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13559f84-7149-47ab-a696-8e2ab2542dc4" path="/var/lib/kubelet/pods/13559f84-7149-47ab-a696-8e2ab2542dc4/volumes"
Jul 17 15:36:22.589750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:22.589717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:36:32.733919 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.733881    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"]
Jul 17 15:36:32.734311 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734296    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13559f84-7149-47ab-a696-8e2ab2542dc4" containerName="maas-api"
Jul 17 15:36:32.734369 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734314    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="13559f84-7149-47ab-a696-8e2ab2542dc4" containerName="maas-api"
Jul 17 15:36:32.734406 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734373    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55cbaac1-7bb4-4fd5-949b-29cb385677c7" containerName="revoke-keys"
Jul 17 15:36:32.734406 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734380    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55cbaac1-7bb4-4fd5-949b-29cb385677c7" containerName="revoke-keys"
Jul 17 15:36:32.734471 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734459    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55cbaac1-7bb4-4fd5-949b-29cb385677c7" containerName="revoke-keys"
Jul 17 15:36:32.734518 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.734471    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="13559f84-7149-47ab-a696-8e2ab2542dc4" containerName="maas-api"
Jul 17 15:36:32.737488 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.737471    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:32.740092 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.740049    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:36:32.746051 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.746027    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"]
Jul 17 15:36:32.843796 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.843756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-818628-x2bzv\" (UID: \"65cfa311-e5ea-49e2-bfc1-c55807f101c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:32.945091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.945036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-818628-x2bzv\" (UID: \"65cfa311-e5ea-49e2-bfc1-c55807f101c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:32.945703 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:32.945677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-818628-x2bzv\" (UID: \"65cfa311-e5ea-49e2-bfc1-c55807f101c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:33.049388 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:33.049302    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:33.188177 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:33.188153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"]
Jul 17 15:36:33.190677 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:36:33.190650    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cfa311_e5ea_49e2_bfc1_c55807f101c9.slice/crio-12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39 WatchSource:0}: Error finding container 12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39: Status 404 returned error can't find the container with id 12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39
Jul 17 15:36:33.607103 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:33.607050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv" event={"ID":"65cfa311-e5ea-49e2-bfc1-c55807f101c9","Type":"ContainerStarted","Data":"31c8252b57b273353f943e3e3613cbfde23f597ebc38b0928200ee679d33e102"}
Jul 17 15:36:33.607266 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:33.607126    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv" event={"ID":"65cfa311-e5ea-49e2-bfc1-c55807f101c9","Type":"ContainerStarted","Data":"12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39"}
Jul 17 15:36:33.623491 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:33.623442    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv" podStartSLOduration=1.6234227460000001 podStartE2EDuration="1.623422746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:36:33.622433941 +0000 UTC m=+2804.189465084" watchObservedRunningTime="2026-07-17 15:36:33.623422746 +0000 UTC m=+2804.190453881"
Jul 17 15:36:34.613733 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:34.613701    2564 generic.go:356] "Generic (PLEG): container finished" podID="65cfa311-e5ea-49e2-bfc1-c55807f101c9" containerID="31c8252b57b273353f943e3e3613cbfde23f597ebc38b0928200ee679d33e102" exitCode=0
Jul 17 15:36:34.614137 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:34.613783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv" event={"ID":"65cfa311-e5ea-49e2-bfc1-c55807f101c9","Type":"ContainerDied","Data":"31c8252b57b273353f943e3e3613cbfde23f597ebc38b0928200ee679d33e102"}
Jul 17 15:36:35.751861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:35.751839    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:35.868987 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:35.868953    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca\") pod \"65cfa311-e5ea-49e2-bfc1-c55807f101c9\" (UID: \"65cfa311-e5ea-49e2-bfc1-c55807f101c9\") "
Jul 17 15:36:35.869354 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:35.869302    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65cfa311-e5ea-49e2-bfc1-c55807f101c9" (UID: "65cfa311-e5ea-49e2-bfc1-c55807f101c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:36:35.970319 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:35.970284    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65cfa311-e5ea-49e2-bfc1-c55807f101c9-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:36.625049 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:36.625021    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv"
Jul 17 15:36:36.625229 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:36.625056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-818628-x2bzv" event={"ID":"65cfa311-e5ea-49e2-bfc1-c55807f101c9","Type":"ContainerDied","Data":"12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39"}
Jul 17 15:36:36.625229 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:36.625132    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12dc368c24d8ca2a774748b52626bd37590193ed7da5b7af609b31e60e81ea39"
Jul 17 15:36:43.400392 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.400355    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:36:43.400907 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.400631    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" podUID="b6bd94b6-1be6-4164-a069-43289040e339" containerName="maas-api" containerID="cri-o://7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709" gracePeriod=30
Jul 17 15:36:43.648233 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.648210    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:36:43.662112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.662035    2564 generic.go:356] "Generic (PLEG): container finished" podID="b6bd94b6-1be6-4164-a069-43289040e339" containerID="7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709" exitCode=0
Jul 17 15:36:43.662112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.662092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" event={"ID":"b6bd94b6-1be6-4164-a069-43289040e339","Type":"ContainerDied","Data":"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"}
Jul 17 15:36:43.662269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.662109    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"
Jul 17 15:36:43.662269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.662144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc" event={"ID":"b6bd94b6-1be6-4164-a069-43289040e339","Type":"ContainerDied","Data":"7231aa7130cd6c0fd9c15376b61a8335d256c8989c8fa74448b666844a14cdfa"}
Jul 17 15:36:43.662269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.662169    2564 scope.go:117] "RemoveContainer" containerID="7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"
Jul 17 15:36:43.672180 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.672152    2564 scope.go:117] "RemoveContainer" containerID="7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"
Jul 17 15:36:43.672474 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:36:43.672451    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709\": container with ID starting with 7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709 not found: ID does not exist" containerID="7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"
Jul 17 15:36:43.672561 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.672485    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709"} err="failed to get container status \"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709\": rpc error: code = NotFound desc = could not find container \"7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709\": container with ID starting with 7e966ea52d33802c0d6d30ee2ba8f3cba7a78af1f24538d9dc9aa0dd88a20709 not found: ID does not exist"
Jul 17 15:36:43.736822 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.736792    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") pod \"b6bd94b6-1be6-4164-a069-43289040e339\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") "
Jul 17 15:36:43.736980 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.736865    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca\") pod \"b6bd94b6-1be6-4164-a069-43289040e339\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") "
Jul 17 15:36:43.736980 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.736932    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmqdn\" (UniqueName: \"kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn\") pod \"b6bd94b6-1be6-4164-a069-43289040e339\" (UID: \"b6bd94b6-1be6-4164-a069-43289040e339\") "
Jul 17 15:36:43.737196 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.737174    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6bd94b6-1be6-4164-a069-43289040e339" (UID: "b6bd94b6-1be6-4164-a069-43289040e339"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:36:43.738984 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.738962    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6bd94b6-1be6-4164-a069-43289040e339" (UID: "b6bd94b6-1be6-4164-a069-43289040e339"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:36:43.739086 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.738963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn" (OuterVolumeSpecName: "kube-api-access-fmqdn") pod "b6bd94b6-1be6-4164-a069-43289040e339" (UID: "b6bd94b6-1be6-4164-a069-43289040e339"). InnerVolumeSpecName "kube-api-access-fmqdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:36:43.838442 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.838411    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmqdn\" (UniqueName: \"kubernetes.io/projected/b6bd94b6-1be6-4164-a069-43289040e339-kube-api-access-fmqdn\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:43.838442 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.838440    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6bd94b6-1be6-4164-a069-43289040e339-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:43.838605 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.838453    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bd94b6-1be6-4164-a069-43289040e339-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:36:43.980148 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.980116    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:36:43.984180 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:43.984156    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-818628-6f99796498-kb2nc"]
Jul 17 15:36:45.951421 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:45.951380    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6bd94b6-1be6-4164-a069-43289040e339" path="/var/lib/kubelet/pods/b6bd94b6-1be6-4164-a069-43289040e339/volumes"
Jul 17 15:36:46.491399 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:36:46.491361    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:37:02.629431 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629396    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:37:02.629863 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629765    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65cfa311-e5ea-49e2-bfc1-c55807f101c9" containerName="revoke-keys"
Jul 17 15:37:02.629863 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629783    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65cfa311-e5ea-49e2-bfc1-c55807f101c9" containerName="revoke-keys"
Jul 17 15:37:02.629863 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629815    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6bd94b6-1be6-4164-a069-43289040e339" containerName="maas-api"
Jul 17 15:37:02.629863 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629822    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6bd94b6-1be6-4164-a069-43289040e339" containerName="maas-api"
Jul 17 15:37:02.630035 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629881    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6bd94b6-1be6-4164-a069-43289040e339" containerName="maas-api"
Jul 17 15:37:02.630035 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.629890    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65cfa311-e5ea-49e2-bfc1-c55807f101c9" containerName="revoke-keys"
Jul 17 15:37:02.632956 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.632940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.635344 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.635322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-c4e0c6-serving-cert\""
Jul 17 15:37:02.635619 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.635602    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jlsxc\""
Jul 17 15:37:02.636317 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.636299    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 15:37:02.636402 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.636314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 15:37:02.645109 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.645084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:37:02.803554 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.803509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74fbb\" (UniqueName: \"kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.803554 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.803567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.803803 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.803629    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.904568 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.904439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74fbb\" (UniqueName: \"kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.904568 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.904523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.904819 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.904589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.905180 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.905159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.907158 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.907140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.924515 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.924475    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74fbb\" (UniqueName: \"kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb\") pod \"maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:02.943430 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:02.943395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:03.103732 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.103696    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:37:03.107212 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:37:03.107184    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72593d2_0ce9_4356_8926_65a8271d91fe.slice/crio-e4e5b2991b46d94ad8d2dc5e6c48accb077ccea30fcb75ce8ea98d705e6bda8d WatchSource:0}: Error finding container e4e5b2991b46d94ad8d2dc5e6c48accb077ccea30fcb75ce8ea98d705e6bda8d: Status 404 returned error can't find the container with id e4e5b2991b46d94ad8d2dc5e6c48accb077ccea30fcb75ce8ea98d705e6bda8d
Jul 17 15:37:03.108551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.108531    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:37:03.775861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.775779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" event={"ID":"a72593d2-0ce9-4356-8926-65a8271d91fe","Type":"ContainerStarted","Data":"26acae6ec5406761b4bbb2de342e6f3d446041b9e8e7107d94e5f91b6c941f0c"}
Jul 17 15:37:03.775861 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.775840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" event={"ID":"a72593d2-0ce9-4356-8926-65a8271d91fe","Type":"ContainerStarted","Data":"e4e5b2991b46d94ad8d2dc5e6c48accb077ccea30fcb75ce8ea98d705e6bda8d"}
Jul 17 15:37:03.776284 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.776008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:03.796964 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:03.796919    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" podStartSLOduration=1.382285092 podStartE2EDuration="1.796906111s" totalImagesPullingTime="414.621019ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:37:03.794893311 +0000 UTC m=+2834.361924444" watchObservedRunningTime="2026-07-17 15:37:03.796906111 +0000 UTC m=+2834.363937244"
Jul 17 15:37:08.383112 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:08.383077    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:37:09.786853 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:09.786822    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:37:10.830992 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.830956    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:37:10.835483 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.835462    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.837999 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.837973    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ae3cc1-serving-cert\""
Jul 17 15:37:10.847292 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.847267    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:37:10.862584 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.862555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.862722 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.862594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.862722 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.862678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.963122 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.963056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.963321 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.963188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.963321 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.963214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.963778 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.963758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.965522 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.965491    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:10.978878 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:10.978850    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc\") pod \"maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:11.148796 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.148718    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:11.304358 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.304323    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:37:11.314881 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:37:11.314852    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1619de82_1ec3_4c2e_98b3_3f5946f26a31.slice/crio-c30ed03254479aa076f78e192d53879fea1f01a43ba877330a0dc996e5d02a6d WatchSource:0}: Error finding container c30ed03254479aa076f78e192d53879fea1f01a43ba877330a0dc996e5d02a6d: Status 404 returned error can't find the container with id c30ed03254479aa076f78e192d53879fea1f01a43ba877330a0dc996e5d02a6d
Jul 17 15:37:11.829651 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.829609    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" event={"ID":"1619de82-1ec3-4c2e-98b3-3f5946f26a31","Type":"ContainerStarted","Data":"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"}
Jul 17 15:37:11.829816 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.829708    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" event={"ID":"1619de82-1ec3-4c2e-98b3-3f5946f26a31","Type":"ContainerStarted","Data":"c30ed03254479aa076f78e192d53879fea1f01a43ba877330a0dc996e5d02a6d"}
Jul 17 15:37:11.829896 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.829849    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:11.855780 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:11.855733    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" podStartSLOduration=1.447467101 podStartE2EDuration="1.855718649s" totalImagesPullingTime="408.251548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:37:11.854058448 +0000 UTC m=+2842.421089617" watchObservedRunningTime="2026-07-17 15:37:11.855718649 +0000 UTC m=+2842.422749782"
Jul 17 15:37:16.189899 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:16.189858    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:37:17.840047 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:17.840016    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:37:28.583337 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.583298    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:37:28.586775 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.586754    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.589438 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.589414    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"openshift-service-ca.crt\""
Jul 17 15:37:28.590376 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.590358    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"auth-test-model-c4e0c6-kserve-self-signed-certs\""
Jul 17 15:37:28.590493 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.590476    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"default-dockercfg-m4h8n\""
Jul 17 15:37:28.590566 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.590506    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"kube-root-ca.crt\""
Jul 17 15:37:28.597161 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.597140    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:37:28.712727 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhtg\" (UniqueName: \"kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.712727 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.712937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.712937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.712937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712882    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.712937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.712905    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814194 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhtg\" (UniqueName: \"kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814812    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814969 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.814969 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.814852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.816750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.816726    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.816917 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.816901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.824557 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.824517    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhtg\" (UniqueName: \"kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg\") pod \"auth-test-model-c4e0c6-kserve-694b44699-52x56\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") " pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:28.897634 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:28.897562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:29.039094 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:29.039037    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:37:29.040368 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:37:29.040346    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15e728cd_d2a0_489d_bd80_9b6a49e6d9c7.slice/crio-978a11c7e88080e9a60b15828ad019d42aa5463fbfefc2b239cefad5ac3c84f2 WatchSource:0}: Error finding container 978a11c7e88080e9a60b15828ad019d42aa5463fbfefc2b239cefad5ac3c84f2: Status 404 returned error can't find the container with id 978a11c7e88080e9a60b15828ad019d42aa5463fbfefc2b239cefad5ac3c84f2
Jul 17 15:37:29.926485 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:29.926443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" event={"ID":"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7","Type":"ContainerStarted","Data":"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"}
Jul 17 15:37:29.926966 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:29.926494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" event={"ID":"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7","Type":"ContainerStarted","Data":"978a11c7e88080e9a60b15828ad019d42aa5463fbfefc2b239cefad5ac3c84f2"}
Jul 17 15:37:29.947294 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:29.947142    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" podStartSLOduration=1.947123253 podStartE2EDuration="1.947123253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:37:29.945894207 +0000 UTC m=+2860.512925358" watchObservedRunningTime="2026-07-17 15:37:29.947123253 +0000 UTC m=+2860.514154389"
Jul 17 15:37:32.488647 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:32.488607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:37:38.898208 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:38.898171    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:38.898660 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:38.898225    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:38.910793 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:38.910765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:38.984926 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:38.984899    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:37:46.713836 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:37:46.713755    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:38:04.182282 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:38:04.182249    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:38:22.690745 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:38:22.690708    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:38:36.380958 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:38:36.380919    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:38:54.443419 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:38:54.443385    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:39:11.389831 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:11.389749    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:39:28.888241 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:28.888200    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:39:43.884602 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:43.884562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:39:50.117513 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:50.117484    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:39:50.123452 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:50.123428    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:39:50.135675 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:50.135652    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:39:50.144024 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:50.144007    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-dhxhn_4deb8705-1e5b-49d4-8daf-329577a363eb/kube-multus/0.log"
Jul 17 15:39:58.740380 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:39:58.740342    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:40:01.788990 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:01.788952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:40:13.760487 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:13.760447    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" podUID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" containerName="main" containerID="cri-o://29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1" gracePeriod=45
Jul 17 15:40:14.018205 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.018145    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:40:14.138391 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138360    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138557 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138422    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138603 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138572    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138642 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138603    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138681 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138639    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lhtg\" (UniqueName: \"kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138681 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138645    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:40:14.138777 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm\") pod \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\" (UID: \"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7\") "
Jul 17 15:40:14.138832 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138812    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home" (OuterVolumeSpecName: "home") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:40:14.138891 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138831    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache" (OuterVolumeSpecName: "model-cache") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:40:14.138981 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138964    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-home\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.139030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138981    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-model-cache\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.139030 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.138992    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tmp-dir\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.140772 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.140747    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg" (OuterVolumeSpecName: "kube-api-access-5lhtg") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "kube-api-access-5lhtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:40:14.140872 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.140778    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm" (OuterVolumeSpecName: "dshm") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:40:14.140872 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.140834    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" (UID: "15e728cd-d2a0-489d-bd80-9b6a49e6d9c7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:40:14.239596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.239561    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-tls-certs\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.239596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.239590    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lhtg\" (UniqueName: \"kubernetes.io/projected/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-kube-api-access-5lhtg\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.239596 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.239600    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7-dshm\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:40:14.808151 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.808115    2564 generic.go:356] "Generic (PLEG): container finished" podID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" containerID="29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1" exitCode=0
Jul 17 15:40:14.808635 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.808159    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"
Jul 17 15:40:14.808635 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.808206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" event={"ID":"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7","Type":"ContainerDied","Data":"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"}
Jul 17 15:40:14.808635 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.808249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56" event={"ID":"15e728cd-d2a0-489d-bd80-9b6a49e6d9c7","Type":"ContainerDied","Data":"978a11c7e88080e9a60b15828ad019d42aa5463fbfefc2b239cefad5ac3c84f2"}
Jul 17 15:40:14.808635 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.808267    2564 scope.go:117] "RemoveContainer" containerID="29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"
Jul 17 15:40:14.818246 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.818225    2564 scope.go:117] "RemoveContainer" containerID="29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"
Jul 17 15:40:14.818548 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:40:14.818529    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1\": container with ID starting with 29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1 not found: ID does not exist" containerID="29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"
Jul 17 15:40:14.818638 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.818562    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1"} err="failed to get container status \"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1\": rpc error: code = NotFound desc = could not find container \"29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1\": container with ID starting with 29be4d89510c10ef64aac38a0cd3535ffb4cec3207e6e4a2fa31dabfa1a775a1 not found: ID does not exist"
Jul 17 15:40:14.830644 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.830625    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:40:14.835482 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:14.835451    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/auth-test-model-c4e0c6-kserve-694b44699-52x56"]
Jul 17 15:40:15.950714 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:15.950679    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" path="/var/lib/kubelet/pods/15e728cd-d2a0-489d-bd80-9b6a49e6d9c7/volumes"
Jul 17 15:40:18.493509 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:18.493470    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:40:34.385763 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:34.385722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:40:56.284609 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:40:56.284570    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:41:15.584640 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:15.584599    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:41:19.388776 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.388738    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:41:19.389311 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.389292    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" containerName="main"
Jul 17 15:41:19.389390 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.389314    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" containerName="main"
Jul 17 15:41:19.389468 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.389455    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15e728cd-d2a0-489d-bd80-9b6a49e6d9c7" containerName="main"
Jul 17 15:41:19.392521 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.392501    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.394848 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.394826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"kube-root-ca.crt\""
Jul 17 15:41:19.395812 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.395793    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"default-dockercfg-m4h8n\""
Jul 17 15:41:19.395918 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.395798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"rate-test-model-c4e0c6-kserve-self-signed-certs\""
Jul 17 15:41:19.395918 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.395796    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c4e0c6\"/\"openshift-service-ca.crt\""
Jul 17 15:41:19.401406 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.401380    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:41:19.407864 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.407842    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.407952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.407879    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.407952 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.407943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.408044 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.407978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.408044 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.408006    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbvqg\" (UniqueName: \"kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.408044 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.408035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509308 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509270    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509632 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509541    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509632 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbvqg\" (UniqueName: \"kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.509835 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.510028 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.510028 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.509913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.511630 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.511610    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.511809 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.511793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.519422 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.519394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbvqg\" (UniqueName: \"kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg\") pod \"rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") " pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.703085 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.702994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:19.841575 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:19.841551    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:41:19.843692 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:41:19.843663    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb39546f5_e14b_42d0_a702_f6b6edb81edc.slice/crio-e3ae30814c166d599f3043c0eea82ac00e231cee78c93e74edbd8d20f99e592c WatchSource:0}: Error finding container e3ae30814c166d599f3043c0eea82ac00e231cee78c93e74edbd8d20f99e592c: Status 404 returned error can't find the container with id e3ae30814c166d599f3043c0eea82ac00e231cee78c93e74edbd8d20f99e592c
Jul 17 15:41:20.161157 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:20.161119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" event={"ID":"b39546f5-e14b-42d0-a702-f6b6edb81edc","Type":"ContainerStarted","Data":"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"}
Jul 17 15:41:20.161157 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:20.161165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" event={"ID":"b39546f5-e14b-42d0-a702-f6b6edb81edc","Type":"ContainerStarted","Data":"e3ae30814c166d599f3043c0eea82ac00e231cee78c93e74edbd8d20f99e592c"}
Jul 17 15:41:20.181996 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:20.181943    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" podStartSLOduration=1.18193011 podStartE2EDuration="1.18193011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:41:20.179641428 +0000 UTC m=+3090.746672559" watchObservedRunningTime="2026-07-17 15:41:20.18193011 +0000 UTC m=+3090.748961243"
Jul 17 15:41:29.703951 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:29.703908    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:29.704583 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:29.703978    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:29.716617 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:29.716590    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:30.226985 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:30.226953    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:41:32.182916 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:32.182875    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:41:48.485890 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:41:48.485856    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:42:17.981884 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:17.981796    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:42:31.880449 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:31.880409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:42:38.698529 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:38.698491    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:42:46.385665 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:46.385630    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:42:53.718493 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:53.718440    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" podUID="b39546f5-e14b-42d0-a702-f6b6edb81edc" containerName="main" containerID="cri-o://60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c" gracePeriod=45
Jul 17 15:42:53.971926 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:53.971874    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:42:54.152893 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.152858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153111 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.152922    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153111 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.152949    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153111 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.152984    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153111 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153011    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153111 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbvqg\" (UniqueName: \"kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg\") pod \"b39546f5-e14b-42d0-a702-f6b6edb81edc\" (UID: \"b39546f5-e14b-42d0-a702-f6b6edb81edc\") "
Jul 17 15:42:54.153384 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153195    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:42:54.153384 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153290    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home" (OuterVolumeSpecName: "home") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:42:54.153384 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153305    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache" (OuterVolumeSpecName: "model-cache") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:42:54.153490 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153417    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-model-cache\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.153490 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153432    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-home\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.153490 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.153441    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-tmp-dir\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.155052 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.155031    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:42:54.155552 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.155534    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm" (OuterVolumeSpecName: "dshm") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 15:42:54.155601 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.155566    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg" (OuterVolumeSpecName: "kube-api-access-hbvqg") pod "b39546f5-e14b-42d0-a702-f6b6edb81edc" (UID: "b39546f5-e14b-42d0-a702-f6b6edb81edc"). InnerVolumeSpecName "kube-api-access-hbvqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:42:54.254243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.254170    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b39546f5-e14b-42d0-a702-f6b6edb81edc-tls-certs\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.254243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.254195    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b39546f5-e14b-42d0-a702-f6b6edb81edc-dshm\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.254243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.254212    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbvqg\" (UniqueName: \"kubernetes.io/projected/b39546f5-e14b-42d0-a702-f6b6edb81edc-kube-api-access-hbvqg\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:54.671971 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.671933    2564 generic.go:356] "Generic (PLEG): container finished" podID="b39546f5-e14b-42d0-a702-f6b6edb81edc" containerID="60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c" exitCode=0
Jul 17 15:42:54.672193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.671984    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"
Jul 17 15:42:54.672193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.672023    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" event={"ID":"b39546f5-e14b-42d0-a702-f6b6edb81edc","Type":"ContainerDied","Data":"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"}
Jul 17 15:42:54.672193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.672076    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx" event={"ID":"b39546f5-e14b-42d0-a702-f6b6edb81edc","Type":"ContainerDied","Data":"e3ae30814c166d599f3043c0eea82ac00e231cee78c93e74edbd8d20f99e592c"}
Jul 17 15:42:54.672193 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.672095    2564 scope.go:117] "RemoveContainer" containerID="60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"
Jul 17 15:42:54.681615 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.681600    2564 scope.go:117] "RemoveContainer" containerID="60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"
Jul 17 15:42:54.681886 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:42:54.681868    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c\": container with ID starting with 60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c not found: ID does not exist" containerID="60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"
Jul 17 15:42:54.681950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.681895    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c"} err="failed to get container status \"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c\": rpc error: code = NotFound desc = could not find container \"60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c\": container with ID starting with 60bb23b6972c11a93f30b6dbed8a5b0a849a3b97b88f2488c4af8aa23446af5c not found: ID does not exist"
Jul 17 15:42:54.695037 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.695018    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:42:54.699673 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:54.699653    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c4e0c6/rate-test-model-c4e0c6-kserve-5bbd797d87-4gjlx"]
Jul 17 15:42:55.507948 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.507915    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7305-140694898902336"]
Jul 17 15:42:55.508379 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.508312    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39546f5-e14b-42d0-a702-f6b6edb81edc" containerName="main"
Jul 17 15:42:55.508379 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.508327    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39546f5-e14b-42d0-a702-f6b6edb81edc" containerName="main"
Jul 17 15:42:55.508463 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.508413    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39546f5-e14b-42d0-a702-f6b6edb81edc" containerName="main"
Jul 17 15:42:55.512841 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.512823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:55.515350 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.515325    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-97hms\""
Jul 17 15:42:55.518400 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.518375    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140694898902336"]
Jul 17 15:42:55.666722 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.666692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khslt\" (UniqueName: \"kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt\") pod \"test-curl-7305-140694898902336\" (UID: \"792ea7e3-d859-4028-be75-07e6a17648df\") " pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:55.767433 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.767352    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khslt\" (UniqueName: \"kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt\") pod \"test-curl-7305-140694898902336\" (UID: \"792ea7e3-d859-4028-be75-07e6a17648df\") " pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:55.776748 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.776720    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khslt\" (UniqueName: \"kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt\") pod \"test-curl-7305-140694898902336\" (UID: \"792ea7e3-d859-4028-be75-07e6a17648df\") " pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:55.824812 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.824786    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:55.951091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.951047    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39546f5-e14b-42d0-a702-f6b6edb81edc" path="/var/lib/kubelet/pods/b39546f5-e14b-42d0-a702-f6b6edb81edc/volumes"
Jul 17 15:42:55.959580 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.959561    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7305-140694898902336"]
Jul 17 15:42:55.961558 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:42:55.961529    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod792ea7e3_d859_4028_be75_07e6a17648df.slice/crio-202bdc58f9349caf1661ca05c1ec3d666fc5fc39966b1acafbf5005315f61cc0 WatchSource:0}: Error finding container 202bdc58f9349caf1661ca05c1ec3d666fc5fc39966b1acafbf5005315f61cc0: Status 404 returned error can't find the container with id 202bdc58f9349caf1661ca05c1ec3d666fc5fc39966b1acafbf5005315f61cc0
Jul 17 15:42:55.962755 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:55.962737    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 15:42:56.684481 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:56.684454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140694898902336" event={"ID":"792ea7e3-d859-4028-be75-07e6a17648df","Type":"ContainerStarted","Data":"202bdc58f9349caf1661ca05c1ec3d666fc5fc39966b1acafbf5005315f61cc0"}
Jul 17 15:42:57.691316 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:57.691278    2564 generic.go:356] "Generic (PLEG): container finished" podID="792ea7e3-d859-4028-be75-07e6a17648df" containerID="5a95b12b8eaff6223f22f44bf341a691166d74a6268c741c7c367ecef8358fa8" exitCode=0
Jul 17 15:42:57.691753 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:57.691330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7305-140694898902336" event={"ID":"792ea7e3-d859-4028-be75-07e6a17648df","Type":"ContainerDied","Data":"5a95b12b8eaff6223f22f44bf341a691166d74a6268c741c7c367ecef8358fa8"}
Jul 17 15:42:58.839674 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.839650    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:58.879266 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.879243    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7305-140694898902336_792ea7e3-d859-4028-be75-07e6a17648df/test-curl-7305-140694898902336/0.log"
Jul 17 15:42:58.892475 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.892453    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khslt\" (UniqueName: \"kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt\") pod \"792ea7e3-d859-4028-be75-07e6a17648df\" (UID: \"792ea7e3-d859-4028-be75-07e6a17648df\") "
Jul 17 15:42:58.894565 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.894539    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt" (OuterVolumeSpecName: "kube-api-access-khslt") pod "792ea7e3-d859-4028-be75-07e6a17648df" (UID: "792ea7e3-d859-4028-be75-07e6a17648df"). InnerVolumeSpecName "kube-api-access-khslt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:42:58.902995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.902971    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7305-140694898902336"]
Jul 17 15:42:58.906760 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.906740    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7305-140694898902336"]
Jul 17 15:42:58.993789 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:58.993700    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khslt\" (UniqueName: \"kubernetes.io/projected/792ea7e3-d859-4028-be75-07e6a17648df-kube-api-access-khslt\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:42:59.096455 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.096420    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"]
Jul 17 15:42:59.096808 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.096796    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792ea7e3-d859-4028-be75-07e6a17648df" containerName="test-curl-7305-140694898902336"
Jul 17 15:42:59.096853 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.096810    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="792ea7e3-d859-4028-be75-07e6a17648df" containerName="test-curl-7305-140694898902336"
Jul 17 15:42:59.096898 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.096889    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="792ea7e3-d859-4028-be75-07e6a17648df" containerName="test-curl-7305-140694898902336"
Jul 17 15:42:59.099945 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.099930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:42:59.102349 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.102328    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:42:59.107838 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.107816    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"]
Jul 17 15:42:59.194675 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.194640    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7\" (UID: \"3728bae1-59f0-4787-a596-6af7cdcd359a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:42:59.295119 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.295003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7\" (UID: \"3728bae1-59f0-4787-a596-6af7cdcd359a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:42:59.295705 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.295682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7\" (UID: \"3728bae1-59f0-4787-a596-6af7cdcd359a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:42:59.412783 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.412753    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:42:59.553427 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.553401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"]
Jul 17 15:42:59.555152 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:42:59.555125    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3728bae1_59f0_4787_a596_6af7cdcd359a.slice/crio-118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f WatchSource:0}: Error finding container 118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f: Status 404 returned error can't find the container with id 118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f
Jul 17 15:42:59.710924 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.710886    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7" event={"ID":"3728bae1-59f0-4787-a596-6af7cdcd359a","Type":"ContainerStarted","Data":"118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f"}
Jul 17 15:42:59.712107 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.712084    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:59.712226 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.712088    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="202bdc58f9349caf1661ca05c1ec3d666fc5fc39966b1acafbf5005315f61cc0"
Jul 17 15:42:59.736276 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:42:59.736250    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7305-140694898902336\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="792ea7e3-d859-4028-be75-07e6a17648df" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:59.948383 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:42:59.948348    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7305-140694898902336\" is forbidden: User \"system:node:ip-10-0-138-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-81.ec2.internal' and this object" podUID="792ea7e3-d859-4028-be75-07e6a17648df" pod="opendatahub/test-curl-7305-140694898902336"
Jul 17 15:42:59.951131 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:42:59.951109    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="792ea7e3-d859-4028-be75-07e6a17648df" path="/var/lib/kubelet/pods/792ea7e3-d859-4028-be75-07e6a17648df/volumes"
Jul 17 15:43:00.718896 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:00.718858    2564 generic.go:356] "Generic (PLEG): container finished" podID="3728bae1-59f0-4787-a596-6af7cdcd359a" containerID="77c61d6a9c3ffc9c4d7aa8c77627df17344d0f990a15fc27c47ae1cc277fac27" exitCode=0
Jul 17 15:43:00.719104 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:00.718896    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7" event={"ID":"3728bae1-59f0-4787-a596-6af7cdcd359a","Type":"ContainerDied","Data":"77c61d6a9c3ffc9c4d7aa8c77627df17344d0f990a15fc27c47ae1cc277fac27"}
Jul 17 15:43:01.856566 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:01.856539    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:43:01.914399 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:01.914368    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca\") pod \"3728bae1-59f0-4787-a596-6af7cdcd359a\" (UID: \"3728bae1-59f0-4787-a596-6af7cdcd359a\") "
Jul 17 15:43:01.914750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:01.914729    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3728bae1-59f0-4787-a596-6af7cdcd359a" (UID: "3728bae1-59f0-4787-a596-6af7cdcd359a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:43:02.015647 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:02.015556    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3728bae1-59f0-4787-a596-6af7cdcd359a-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:02.730710 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:02.730677    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7"
Jul 17 15:43:02.730710 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:02.730702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7" event={"ID":"3728bae1-59f0-4787-a596-6af7cdcd359a","Type":"ContainerDied","Data":"118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f"}
Jul 17 15:43:02.730937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:02.730752    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="118edcf78142464c51c0a354f6c3347f3eda5d12476fc805c6afc2c66234605f"
Jul 17 15:43:09.591461 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.591421    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:43:09.591981 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.591694    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerName="maas-api" containerID="cri-o://26acae6ec5406761b4bbb2de342e6f3d446041b9e8e7107d94e5f91b6c941f0c" gracePeriod=30
Jul 17 15:43:09.769731 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.769691    2564 generic.go:356] "Generic (PLEG): container finished" podID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerID="26acae6ec5406761b4bbb2de342e6f3d446041b9e8e7107d94e5f91b6c941f0c" exitCode=0
Jul 17 15:43:09.769906 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.769733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" event={"ID":"a72593d2-0ce9-4356-8926-65a8271d91fe","Type":"ContainerDied","Data":"26acae6ec5406761b4bbb2de342e6f3d446041b9e8e7107d94e5f91b6c941f0c"}
Jul 17 15:43:09.851283 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.851226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:43:09.868496 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.868470    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74fbb\" (UniqueName: \"kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb\") pod \"a72593d2-0ce9-4356-8926-65a8271d91fe\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") "
Jul 17 15:43:09.868611 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.868544    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca\") pod \"a72593d2-0ce9-4356-8926-65a8271d91fe\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") "
Jul 17 15:43:09.868611 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.868585    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls\") pod \"a72593d2-0ce9-4356-8926-65a8271d91fe\" (UID: \"a72593d2-0ce9-4356-8926-65a8271d91fe\") "
Jul 17 15:43:09.868961 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.868934    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a72593d2-0ce9-4356-8926-65a8271d91fe" (UID: "a72593d2-0ce9-4356-8926-65a8271d91fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:43:09.870452 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.870429    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a72593d2-0ce9-4356-8926-65a8271d91fe" (UID: "a72593d2-0ce9-4356-8926-65a8271d91fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:43:09.870587 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.870566    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb" (OuterVolumeSpecName: "kube-api-access-74fbb") pod "a72593d2-0ce9-4356-8926-65a8271d91fe" (UID: "a72593d2-0ce9-4356-8926-65a8271d91fe"). InnerVolumeSpecName "kube-api-access-74fbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:43:09.969955 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.969928    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a72593d2-0ce9-4356-8926-65a8271d91fe-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:09.969955 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.969955    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a72593d2-0ce9-4356-8926-65a8271d91fe-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:09.970145 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:09.969969    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74fbb\" (UniqueName: \"kubernetes.io/projected/a72593d2-0ce9-4356-8926-65a8271d91fe-kube-api-access-74fbb\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:10.777360 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:10.777322    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" event={"ID":"a72593d2-0ce9-4356-8926-65a8271d91fe","Type":"ContainerDied","Data":"e4e5b2991b46d94ad8d2dc5e6c48accb077ccea30fcb75ce8ea98d705e6bda8d"}
Jul 17 15:43:10.777360 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:10.777363    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"
Jul 17 15:43:10.777853 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:10.777420    2564 scope.go:117] "RemoveContainer" containerID="26acae6ec5406761b4bbb2de342e6f3d446041b9e8e7107d94e5f91b6c941f0c"
Jul 17 15:43:10.800097 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:10.800054    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:43:10.802048 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:10.802027    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk"]
Jul 17 15:43:11.950916 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:11.950881    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" path="/var/lib/kubelet/pods/a72593d2-0ce9-4356-8926-65a8271d91fe/volumes"
Jul 17 15:43:12.780846 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:12.780800    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c4e0c6-8fb84dbfb-xwxgk" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.76:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 17 15:43:13.285156 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:13.285122    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:43:25.237028 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.236982    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"]
Jul 17 15:43:25.237617 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237595    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerName="maas-api"
Jul 17 15:43:25.237704 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237620    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerName="maas-api"
Jul 17 15:43:25.237704 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237651    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3728bae1-59f0-4787-a596-6af7cdcd359a" containerName="revoke-keys"
Jul 17 15:43:25.237704 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237660    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3728bae1-59f0-4787-a596-6af7cdcd359a" containerName="revoke-keys"
Jul 17 15:43:25.237870 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237760    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3728bae1-59f0-4787-a596-6af7cdcd359a" containerName="revoke-keys"
Jul 17 15:43:25.237870 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.237774    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a72593d2-0ce9-4356-8926-65a8271d91fe" containerName="maas-api"
Jul 17 15:43:25.242255 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.242233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:25.244768 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.244748    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qn2v2\""
Jul 17 15:43:25.248943 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.248901    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"]
Jul 17 15:43:25.301460 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.301434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9\" (UID: \"c8d04229-c930-4012-a34b-6785995b913b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:25.401821 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.401788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9\" (UID: \"c8d04229-c930-4012-a34b-6785995b913b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:25.402415 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.402395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9\" (UID: \"c8d04229-c930-4012-a34b-6785995b913b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:25.554249 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.554175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:25.710771 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.710744    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"]
Jul 17 15:43:25.713535 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:43:25.713501    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d04229_c930_4012_a34b_6785995b913b.slice/crio-71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0 WatchSource:0}: Error finding container 71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0: Status 404 returned error can't find the container with id 71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0
Jul 17 15:43:25.862851 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:25.862815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9" event={"ID":"c8d04229-c930-4012-a34b-6785995b913b","Type":"ContainerStarted","Data":"71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0"}
Jul 17 15:43:26.869639 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:26.869606    2564 generic.go:356] "Generic (PLEG): container finished" podID="c8d04229-c930-4012-a34b-6785995b913b" containerID="bd115e801bf68c01359880820fba2fee4bc1ec1fcd62003be1394fcd9bb981f4" exitCode=0
Jul 17 15:43:26.870013 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:26.869694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9" event={"ID":"c8d04229-c930-4012-a34b-6785995b913b","Type":"ContainerDied","Data":"bd115e801bf68c01359880820fba2fee4bc1ec1fcd62003be1394fcd9bb981f4"}
Jul 17 15:43:28.008545 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.008524    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:28.129183 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.129100    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca\") pod \"c8d04229-c930-4012-a34b-6785995b913b\" (UID: \"c8d04229-c930-4012-a34b-6785995b913b\") "
Jul 17 15:43:28.129418 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.129397    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8d04229-c930-4012-a34b-6785995b913b" (UID: "c8d04229-c930-4012-a34b-6785995b913b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:43:28.229680 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.229645    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8d04229-c930-4012-a34b-6785995b913b-maas-api-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:28.881212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.881181    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9"
Jul 17 15:43:28.881387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.881215    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9" event={"ID":"c8d04229-c930-4012-a34b-6785995b913b","Type":"ContainerDied","Data":"71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0"}
Jul 17 15:43:28.881387 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:28.881262    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71c7dd3f5e0a96cdf299c0c9998d7a39f1cac0ae2771265e739325f1b90fa0a0"
Jul 17 15:43:35.750517 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:35.750477    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:43:35.751075 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:35.750788    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" podUID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" containerName="maas-api" containerID="cri-o://ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0" gracePeriod=30
Jul 17 15:43:36.201753 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.201729    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:43:36.300110 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.300010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca\") pod \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") "
Jul 17 15:43:36.300254 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.300132    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc\") pod \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") "
Jul 17 15:43:36.300297 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.300249    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls\") pod \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\" (UID: \"1619de82-1ec3-4c2e-98b3-3f5946f26a31\") "
Jul 17 15:43:36.300434 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.300411    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1619de82-1ec3-4c2e-98b3-3f5946f26a31" (UID: "1619de82-1ec3-4c2e-98b3-3f5946f26a31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 15:43:36.300577 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.300554    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1619de82-1ec3-4c2e-98b3-3f5946f26a31-openshift-service-ca\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:36.302184 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.302158    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1619de82-1ec3-4c2e-98b3-3f5946f26a31" (UID: "1619de82-1ec3-4c2e-98b3-3f5946f26a31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 15:43:36.302269 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.302238    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc" (OuterVolumeSpecName: "kube-api-access-j6hrc") pod "1619de82-1ec3-4c2e-98b3-3f5946f26a31" (UID: "1619de82-1ec3-4c2e-98b3-3f5946f26a31"). InnerVolumeSpecName "kube-api-access-j6hrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:43:36.401280 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.401249    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/1619de82-1ec3-4c2e-98b3-3f5946f26a31-kube-api-access-j6hrc\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:36.401280 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.401276    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1619de82-1ec3-4c2e-98b3-3f5946f26a31-maas-api-tls\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:36.926750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.926713    2564 generic.go:356] "Generic (PLEG): container finished" podID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" containerID="ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0" exitCode=0
Jul 17 15:43:36.927212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.926757    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"
Jul 17 15:43:36.927212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.926765    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" event={"ID":"1619de82-1ec3-4c2e-98b3-3f5946f26a31","Type":"ContainerDied","Data":"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"}
Jul 17 15:43:36.927212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.926836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4" event={"ID":"1619de82-1ec3-4c2e-98b3-3f5946f26a31","Type":"ContainerDied","Data":"c30ed03254479aa076f78e192d53879fea1f01a43ba877330a0dc996e5d02a6d"}
Jul 17 15:43:36.927212 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.926872    2564 scope.go:117] "RemoveContainer" containerID="ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"
Jul 17 15:43:36.936586 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.936566    2564 scope.go:117] "RemoveContainer" containerID="ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"
Jul 17 15:43:36.936813 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:43:36.936796    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0\": container with ID starting with ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0 not found: ID does not exist" containerID="ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"
Jul 17 15:43:36.936856 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.936823    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0"} err="failed to get container status \"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0\": rpc error: code = NotFound desc = could not find container \"ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0\": container with ID starting with ffd1bb23454f2db559fb12916bee0ef8a4757cc6ef41ca039da6c7308b663bb0 not found: ID does not exist"
Jul 17 15:43:36.950506 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.950484    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:43:36.954866 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:36.954843    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae3cc1-76ff4b75b8-rkhf4"]
Jul 17 15:43:37.950386 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:37.950354    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" path="/var/lib/kubelet/pods/1619de82-1ec3-4c2e-98b3-3f5946f26a31/volumes"
Jul 17 15:43:38.893216 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:38.893182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bvnbh"]
Jul 17 15:43:45.636551 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636467    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636860    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" containerName="maas-api"
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636871    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" containerName="maas-api"
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636893    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d04229-c930-4012-a34b-6785995b913b" containerName="revoke-keys"
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636902    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d04229-c930-4012-a34b-6785995b913b" containerName="revoke-keys"
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636964    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8d04229-c930-4012-a34b-6785995b913b" containerName="revoke-keys"
Jul 17 15:43:45.636995 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.636977    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1619de82-1ec3-4c2e-98b3-3f5946f26a31" containerName="maas-api"
Jul 17 15:43:45.641371 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.641348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:45.653310 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.653287    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:45.782180 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.782144    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn572\" (UniqueName: \"kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572\") pod \"kuadrant-operator-catalog-7t68g\" (UID: \"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd\") " pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:45.882717 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.882681    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn572\" (UniqueName: \"kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572\") pod \"kuadrant-operator-catalog-7t68g\" (UID: \"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd\") " pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:45.891736 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.891680    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn572\" (UniqueName: \"kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572\") pod \"kuadrant-operator-catalog-7t68g\" (UID: \"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd\") " pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:45.952026 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:45.951999    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:46.087834 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:46.087799    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:46.090277 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:43:46.090245    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be4aab9_3e8e_4d0f_bc8b_2e5a48505ecd.slice/crio-6fdfed3f4d8e268dd72049f4e7ceebdb73f6f9a38e5d8b845d1b8c2ed7e2a359 WatchSource:0}: Error finding container 6fdfed3f4d8e268dd72049f4e7ceebdb73f6f9a38e5d8b845d1b8c2ed7e2a359: Status 404 returned error can't find the container with id 6fdfed3f4d8e268dd72049f4e7ceebdb73f6f9a38e5d8b845d1b8c2ed7e2a359
Jul 17 15:43:46.982975 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:46.982936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" event={"ID":"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd","Type":"ContainerStarted","Data":"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"}
Jul 17 15:43:46.983356 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:46.983003    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" event={"ID":"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd","Type":"ContainerStarted","Data":"6fdfed3f4d8e268dd72049f4e7ceebdb73f6f9a38e5d8b845d1b8c2ed7e2a359"}
Jul 17 15:43:47.004338 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:47.004288    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" podStartSLOduration=1.6290802709999999 podStartE2EDuration="2.004272131s" totalImagesPullingTime="375.19186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 15:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:43:47.001640179 +0000 UTC m=+3237.568671311" watchObservedRunningTime="2026-07-17 15:43:47.004272131 +0000 UTC m=+3237.571303262"
Jul 17 15:43:51.624157 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:51.624124    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84c6465864-kx76v_1846718a-89e4-47ff-a25e-b29f73e157a2/authorino/0.log"
Jul 17 15:43:55.952276 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:55.952242    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:55.952685 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:55.952337    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:55.974373 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:55.974348    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:56.060144 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:56.060118    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:56.203555 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:56.203470    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:57.017218 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.017190    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4px7w_77fa29ed-3d9f-45aa-bf56-d2899b788186/manager/0.log"
Jul 17 15:43:57.133492 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.133459    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6dzdn_584db22e-ee21-4b1b-9015-847365f25090/manager/0.log"
Jul 17 15:43:57.359235 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.359209    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xx49w_95147308-7c75-4eca-a1d1-e0104f7ec585/server/0.log"
Jul 17 15:43:57.474337 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.474311    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bz2bj_fe2fce4c-57de-4432-affe-e6de27ac9166/manager/0.log"
Jul 17 15:43:57.580431 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.580409    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcf5668f8-bljrz_f94d6863-9b80-4d79-97e6-7813d72a4ac8/copy-manifests/0.log"
Jul 17 15:43:57.593822 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:57.593802    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcf5668f8-bljrz_f94d6863-9b80-4d79-97e6-7813d72a4ac8/manager/0.log"
Jul 17 15:43:58.047945 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.047910    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" podUID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" containerName="registry-server" containerID="cri-o://ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66" gracePeriod=2
Jul 17 15:43:58.297457 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.297432    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:58.306003 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.305955    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738355-75f5f_f51fac8e-8cfd-4f54-9b18-9400b3458139/cleanup/0.log"
Jul 17 15:43:58.387357 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.387332    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn572\" (UniqueName: \"kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572\") pod \"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd\" (UID: \"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd\") "
Jul 17 15:43:58.389365 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.389342    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572" (OuterVolumeSpecName: "kube-api-access-dn572") pod "5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" (UID: "5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd"). InnerVolumeSpecName "kube-api-access-dn572". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 15:43:58.414149 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.414126    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738370-xhxbc_bd0e62dd-690e-4e47-bdde-b5669a36839f/cleanup/0.log"
Jul 17 15:43:58.488323 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.488283    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn572\" (UniqueName: \"kubernetes.io/projected/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd-kube-api-access-dn572\") on node \"ip-10-0-138-81.ec2.internal\" DevicePath \"\""
Jul 17 15:43:58.516500 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.516475    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-c30485-6pctp_55cbaac1-7bb4-4fd5-949b-29cb385677c7/revoke-keys/0.log"
Jul 17 15:43:58.620958 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.620934    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-818628-x2bzv_65cfa311-e5ea-49e2-bfc1-c55807f101c9/revoke-keys/0.log"
Jul 17 15:43:58.730854 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.730829    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-4120cb-ldc46_816a1ae9-f7d1-44a0-bfdb-ac6789c54e5a/revoke-keys/0.log"
Jul 17 15:43:58.840142 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.840117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-c10dc4-vj99c_0855e14a-a49f-4ae7-bb0e-c8759805e840/revoke-keys/0.log"
Jul 17 15:43:58.948091 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:58.948001    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-a502a35a-cd898_50002337-2571-4a85-aa51-c9f52263d98e/revoke-keys/0.log"
Jul 17 15:43:59.054750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.054717    2564 generic.go:356] "Generic (PLEG): container finished" podID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" containerID="ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66" exitCode=0
Jul 17 15:43:59.054750 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.054755    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7t68g"
Jul 17 15:43:59.055268 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.054806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" event={"ID":"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd","Type":"ContainerDied","Data":"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"}
Jul 17 15:43:59.055268 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.054856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7t68g" event={"ID":"5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd","Type":"ContainerDied","Data":"6fdfed3f4d8e268dd72049f4e7ceebdb73f6f9a38e5d8b845d1b8c2ed7e2a359"}
Jul 17 15:43:59.055268 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.054881    2564 scope.go:117] "RemoveContainer" containerID="ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"
Jul 17 15:43:59.055505 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.055484    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-afffcc69-8sdwx_72cba91e-74f3-4b93-b009-cd5755be1353/revoke-keys/0.log"
Jul 17 15:43:59.064799 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.064776    2564 scope.go:117] "RemoveContainer" containerID="ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"
Jul 17 15:43:59.065035 ip-10-0-138-81 kubenswrapper[2564]: E0717 15:43:59.065017    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66\": container with ID starting with ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66 not found: ID does not exist" containerID="ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"
Jul 17 15:43:59.065137 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.065050    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66"} err="failed to get container status \"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66\": rpc error: code = NotFound desc = could not find container \"ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66\": container with ID starting with ac4b4dbb51b79ef6e986320c64e95f728d7c233eb7a22cc71f0dbfcc0db62b66 not found: ID does not exist"
Jul 17 15:43:59.080317 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.080275    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:59.082381 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.082357    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7t68g"]
Jul 17 15:43:59.165249 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.165224    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-c4e0c6-fkpd7_3728bae1-59f0-4787-a596-6af7cdcd359a/revoke-keys/0.log"
Jul 17 15:43:59.268368 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.268298    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-ae3cc1-72vj9_c8d04229-c930-4012-a34b-6785995b913b/revoke-keys/0.log"
Jul 17 15:43:59.951972 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:43:59.951929    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" path="/var/lib/kubelet/pods/5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd/volumes"
Jul 17 15:44:00.625867 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:00.625839    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84c6465864-kx76v_1846718a-89e4-47ff-a25e-b29f73e157a2/authorino/0.log"
Jul 17 15:44:00.854039 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:00.854015    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4vpnh_cf917cbb-eec7-4eb7-ab60-e3213d93e12e/manager/0.log"
Jul 17 15:44:00.959647 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:00.959575    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6zcms_be659dc6-56b6-40f4-b054-bf7432769ba2/kuadrant-console-plugin/0.log"
Jul 17 15:44:01.076164 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:01.076129    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qpbdr_e37c9269-9fc3-4355-b79c-b69f47e59416/registry-server/0.log"
Jul 17 15:44:01.193943 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:01.193913    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vp5w8_1db69919-4131-45a5-9b37-f00857b5fd3b/manager/0.log"
Jul 17 15:44:01.299825 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:01.299754    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bvnbh_21f66bd4-4c7c-458a-83a3-d0e4af585e4b/limitador/0.log"
Jul 17 15:44:01.409060 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:01.409030    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-44dv5_5e92c85c-81a6-46d0-ac5d-941ce155a782/manager/0.log"
Jul 17 15:44:02.075865 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:02.075828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cdb8d648-nn7d2_8c1f7d77-954c-47d4-9f1a-78b0ee2912be/kube-auth-proxy/0.log"
Jul 17 15:44:02.539822 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:02.539738    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4d4bdd7d-ln92r_f5eb9ce6-f7ca-454c-900b-76b4cb6cae20/router/0.log"
Jul 17 15:44:02.969357 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:02.969326    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nkhq8_5fc67c78-9e12-43c0-a55d-a5614dea757d/main/0.log"
Jul 17 15:44:11.365059 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:11.365034    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6hdkd_a3f0b694-edc9-4e75-81de-7b528fa2195f/global-pull-secret-syncer/0.log"
Jul 17 15:44:11.521232 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:11.521200    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mtglb_e79ace2d-20ee-49ab-8d5d-c465f24d0c76/konnectivity-agent/0.log"
Jul 17 15:44:11.585950 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:11.585919    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-81.ec2.internal_5708bb1a977bc61e2607d3ed9ee9c042/haproxy/0.log"
Jul 17 15:44:17.076835 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.076800    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-84c6465864-kx76v_1846718a-89e4-47ff-a25e-b29f73e157a2/authorino/0.log"
Jul 17 15:44:17.153391 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.153362    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4vpnh_cf917cbb-eec7-4eb7-ab60-e3213d93e12e/manager/0.log"
Jul 17 15:44:17.183051 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.183028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6zcms_be659dc6-56b6-40f4-b054-bf7432769ba2/kuadrant-console-plugin/0.log"
Jul 17 15:44:17.229478 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.229455    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-qpbdr_e37c9269-9fc3-4355-b79c-b69f47e59416/registry-server/0.log"
Jul 17 15:44:17.339715 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.339643    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vp5w8_1db69919-4131-45a5-9b37-f00857b5fd3b/manager/0.log"
Jul 17 15:44:17.381218 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.381197    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bvnbh_21f66bd4-4c7c-458a-83a3-d0e4af585e4b/limitador/0.log"
Jul 17 15:44:17.445819 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:17.445796    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-44dv5_5e92c85c-81a6-46d0-ac5d-941ce155a782/manager/0.log"
Jul 17 15:44:19.383656 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:19.383624    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-l5xqp_f8126ee2-942d-4fea-a824-2e251bcf23b0/cluster-monitoring-operator/0.log"
Jul 17 15:44:19.888937 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:19.888910    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8tjt_06533c3f-ba49-462e-b824-9f5b0e28ac3b/node-exporter/0.log"
Jul 17 15:44:19.914949 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:19.914921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8tjt_06533c3f-ba49-462e-b824-9f5b0e28ac3b/kube-rbac-proxy/0.log"
Jul 17 15:44:19.942535 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:19.942507    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x8tjt_06533c3f-ba49-462e-b824-9f5b0e28ac3b/init-textfile/0.log"
Jul 17 15:44:21.593818 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.593782    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"]
Jul 17 15:44:21.594291 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.594172    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" containerName="registry-server"
Jul 17 15:44:21.594291 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.594184    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" containerName="registry-server"
Jul 17 15:44:21.594291 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.594254    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be4aab9-3e8e-4d0f-bc8b-2e5a48505ecd" containerName="registry-server"
Jul 17 15:44:21.597570 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.597553    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.600011 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.599991    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8rfwj\"/\"kube-root-ca.crt\""
Jul 17 15:44:21.600142 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.600119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8rfwj\"/\"default-dockercfg-q4hr6\""
Jul 17 15:44:21.600800 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.600783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8rfwj\"/\"openshift-service-ca.crt\""
Jul 17 15:44:21.607054 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.607030    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"]
Jul 17 15:44:21.681991 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.681943    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-podres\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.681991 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.682001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-proc\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.682262 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.682019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2p5q\" (UniqueName: \"kubernetes.io/projected/54d1b0fb-e795-4220-a436-c5240c5fd421-kube-api-access-v2p5q\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.682262 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.682054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-lib-modules\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.682262 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.682104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-sys\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.769126 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.769100    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-lkgf2_bc284a53-0080-4ac6-9e8a-a46e170d7a87/networking-console-plugin/0.log"
Jul 17 15:44:21.783231 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-proc\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783333 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2p5q\" (UniqueName: \"kubernetes.io/projected/54d1b0fb-e795-4220-a436-c5240c5fd421-kube-api-access-v2p5q\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783333 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-lib-modules\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783333 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783310    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-proc\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783333 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783323    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-sys\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783494 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-podres\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783494 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-sys\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783494 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-lib-modules\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.783593 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.783502    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54d1b0fb-e795-4220-a436-c5240c5fd421-podres\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.791692 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.791663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2p5q\" (UniqueName: \"kubernetes.io/projected/54d1b0fb-e795-4220-a436-c5240c5fd421-kube-api-access-v2p5q\") pod \"perf-node-gather-daemonset-qvjv2\" (UID: \"54d1b0fb-e795-4220-a436-c5240c5fd421\") " pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:21.907708 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:21.907614    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:22.049191 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.049167    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"]
Jul 17 15:44:22.051409 ip-10-0-138-81 kubenswrapper[2564]: W0717 15:44:22.051376    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54d1b0fb_e795_4220_a436_c5240c5fd421.slice/crio-510053ee7330d288604d309cab3ab3af251eb0d0bc9d3ac817685cf35edffafc WatchSource:0}: Error finding container 510053ee7330d288604d309cab3ab3af251eb0d0bc9d3ac817685cf35edffafc: Status 404 returned error can't find the container with id 510053ee7330d288604d309cab3ab3af251eb0d0bc9d3ac817685cf35edffafc
Jul 17 15:44:22.174173 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.174054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2" event={"ID":"54d1b0fb-e795-4220-a436-c5240c5fd421","Type":"ContainerStarted","Data":"45a2fd5ee10a71865d562846c5da6a62a5144b52817f1e37c5d2583f485ef290"}
Jul 17 15:44:22.174173 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.174135    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2" event={"ID":"54d1b0fb-e795-4220-a436-c5240c5fd421","Type":"ContainerStarted","Data":"510053ee7330d288604d309cab3ab3af251eb0d0bc9d3ac817685cf35edffafc"}
Jul 17 15:44:22.174404 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.174269    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:22.193721 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.193672    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2" podStartSLOduration=1.1936556010000001 podStartE2EDuration="1.193655601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 15:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 15:44:22.191280274 +0000 UTC m=+3272.758311426" watchObservedRunningTime="2026-07-17 15:44:22.193655601 +0000 UTC m=+3272.760686733"
Jul 17 15:44:22.373297 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.373268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/1.log"
Jul 17 15:44:22.378238 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:22.378217    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-jlhcq_ddc5821f-f2b8-4072-9b11-abf6a3ac74ac/console-operator/2.log"
Jul 17 15:44:23.471287 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:23.471252    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-xv295_faed7fae-93eb-4f34-829c-af921104356d/volume-data-source-validator/0.log"
Jul 17 15:44:24.674868 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:24.674817    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-txmxh_0fcace03-2a71-44e3-9bea-c33d3b7c4c04/dns/0.log"
Jul 17 15:44:24.700411 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:24.700385    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-txmxh_0fcace03-2a71-44e3-9bea-c33d3b7c4c04/kube-rbac-proxy/0.log"
Jul 17 15:44:24.728486 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:24.728455    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bvflg_69b566a4-7ec7-483b-b348-2a7b08dcceef/dns-node-resolver/0.log"
Jul 17 15:44:25.421209 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:25.421179    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qmhfb_93e485cf-83b7-43d4-99bd-d9882c233d5b/node-ca/0.log"
Jul 17 15:44:26.703638 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:26.703611    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cdb8d648-nn7d2_8c1f7d77-954c-47d4-9f1a-78b0ee2912be/kube-auth-proxy/0.log"
Jul 17 15:44:26.868826 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:26.868801    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4d4bdd7d-ln92r_f5eb9ce6-f7ca-454c-900b-76b4cb6cae20/router/0.log"
Jul 17 15:44:27.614696 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:27.614661    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rx4wv_a306a5b7-21a5-4135-9294-e48b5fb6793a/serve-healthcheck-canary/0.log"
Jul 17 15:44:28.151396 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.151365    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ls4vc_26575325-fed3-4634-8f77-278b4bd35d45/insights-operator/0.log"
Jul 17 15:44:28.152243 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.152223    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ls4vc_26575325-fed3-4634-8f77-278b4bd35d45/insights-operator/1.log"
Jul 17 15:44:28.189388 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.189365    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8rfwj/perf-node-gather-daemonset-qvjv2"
Jul 17 15:44:28.258746 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.258710    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcnmv_9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34/kube-rbac-proxy/0.log"
Jul 17 15:44:28.283589 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.283568    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcnmv_9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34/exporter/0.log"
Jul 17 15:44:28.308526 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:28.308506    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcnmv_9ed7b49d-c3a0-4eaf-bf3a-54405c2c5c34/extractor/0.log"
Jul 17 15:44:30.752902 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:30.752873    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4px7w_77fa29ed-3d9f-45aa-bf56-d2899b788186/manager/0.log"
Jul 17 15:44:30.884333 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:30.884305    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-6dzdn_584db22e-ee21-4b1b-9015-847365f25090/manager/0.log"
Jul 17 15:44:31.070374 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:31.070342    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-xx49w_95147308-7c75-4eca-a1d1-e0104f7ec585/server/0.log"
Jul 17 15:44:31.143663 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:31.143631    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bz2bj_fe2fce4c-57de-4432-affe-e6de27ac9166/manager/0.log"
Jul 17 15:44:31.221452 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:31.221422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcf5668f8-bljrz_f94d6863-9b80-4d79-97e6-7813d72a4ac8/manager/0.log"
Jul 17 15:44:31.240090 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:31.240045    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dcf5668f8-bljrz_f94d6863-9b80-4d79-97e6-7813d72a4ac8/copy-manifests/0.log"
Jul 17 15:44:32.650382 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:32.650355    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cdb8d648-nn7d2_8c1f7d77-954c-47d4-9f1a-78b0ee2912be/kube-auth-proxy/0.log"
Jul 17 15:44:32.788938 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:32.788905    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4d4bdd7d-ln92r_f5eb9ce6-f7ca-454c-900b-76b4cb6cae20/router/0.log"
Jul 17 15:44:34.304577 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:34.304515    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56cdb8d648-nn7d2_8c1f7d77-954c-47d4-9f1a-78b0ee2912be/kube-auth-proxy/0.log"
Jul 17 15:44:34.442954 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:34.442927    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c4d4bdd7d-ln92r_f5eb9ce6-f7ca-454c-900b-76b4cb6cae20/router/0.log"
Jul 17 15:44:38.408506 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:38.408472    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7j5w7_a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c/kube-storage-version-migrator-operator/1.log"
Jul 17 15:44:38.409557 ip-10-0-138-81 kubenswrapper[2564]: I0717 15:44:38.409538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-7j5w7_a983fcd7-fb3b-4bd7-a2de-ef61c3afa35c/kube-storage-version-migrator-operator/0.log"